Human  Factors  Engineering  Evaluation  of  the  CH-47F 
Horizontal  Situation  Display  Hover 

by  Jared  Sapp,  Sage  Jessee,  Jonathan  Crutcher,  Mary  Carolyn  King,  and 

Anthony  Morris 


ARL-TR-5882 


January  2012 


Approved  for  public  release;  distribution  is  unlimited. 


NOTICES 

Disclaimers 

The  findings  in  this  report  are  not  to  be  construed  as  an  official  Department  of  the  Army  position  unless 
so  designated  by  other  authorized  documents. 

Citation  of  manufacturer’s  or  trade  names  does  not  constitute  an  official  endorsement  or  approval  of  the 
use  thereof. 


Destroy  this  report  when  it  is  no  longer  needed.  Do  not  return  it  to  the  originator. 


Army  Research  Laboratory 

Aberdeen  Proving  Ground,  MD  21005-5425 


ARL-TR-5882 


January  2012 


Human  Factors  Engineering  Evaluation  of  the  CH-47F 
Horizontal  Situation  Display  Hover 


Jared  Sapp,  Sage  Jessee,  Jonathan  Crutcher,  Mary  Carolyn  King, 

and  Anthony  Morris 

Human  Research  and  Engineering  Directorate,  ARL 


Approved  for  public  release;  distribution  is  unlimited. 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering 
and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  the  burden,  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson 
Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to 
comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  OMB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

A  crew  station  working  group  demonstration  was  conducted  on  the  manual  flight  control  effectiveness  and  usability  of  the 
simulated  CH-47F  Horizontal  Situation  Display  Hover  (HSDH)  pilot  visual  interface.  The  study  compared  three  HSDH  display 
configurations:  (1)  the  existing  CH-47F  baseline  HSDH,  (2)  a  proposed  HSDH  redesign,  and  (3)  a  default  decluttered 
configuration  of  the  proposed  HSDH  redesign.  Eight  career  pilots  evaluated  the  HSDH  usability  under  simulated  brownout 
visual  conditions  for  six  different  hover  and  landing  tasks.  Usability  was  assessed  along  four  dimensions,  including  cockpit 
visual  gaze,  aircraft  control,  pilot  workload,  and  pilot  vehicle  interface  preference  for  ease  of  use  and  effectiveness  of 
information  presentation.  Crew  gaze  analyses  converged  to  indicate  that  pilots  had  a  collective  reduction  in  the  amount  of 
visual  workload  for  the  proposed  and  decluttered  HSDH  compared  to  the  baseline.  Aircraft  control  analyses  indicated  that 
pilots  had  gains  in  the  overall  aviating  effectiveness  while  flying  with  the  proposed  and  decluttered  HSDH.  Crews  reported  that 
workload  demands  were  at  acceptable  levels  while  using  the  new  displays  during  all  mission  tasks  included  in  the  study.  Pilot 
feedback  indicated  that  the  interface  readability  of  the  new  displays  was  effectively  designed  and  presented.  Pilots  highly 
recommended  the  overall  design  of  the  proposed  HSDH  and  voted  unanimously  for  its  immediate  implementation. 

15.  SUBJECT  TERMS 

CH-47F,  human  factors,  brownout,  HSDH 

17.  LIMITATION  13.  NUMBER 
OF  ABSTRACT  OF  PAGES 

UU _ 112 

Standard  Form  298  (Rev.  8/98) 

Prescribed  by  ANSI  Std.  Z39.18 


19a.  NAME  OF  RESPONSIBLE  PERSON 

Jared  Sapp 

19b.  TELEPHONE  NUMBER  (Include  area  code) 

256-824-7937 


16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 

b.  ABSTRACT 

C.  THIS  PAGE 

Unclassified 

Unclassified 

Unclassified 

1.  REPORT  DATE  (DD-MM-YYYY) 

2.  REPORT  TYPE 

January  2012 

Final 

4.  TITLE  AND  SUBTITLE 

Human  Factors  Engineering  Evaluation  of  the  CH-47F  Horizontal  Situation 
Display  Hover 


6.  AUTHOR(S) 

Jared  Sapp,  Sage  Jessee,  Jonathan  Crutcher,  Mary  Carolyn  King,  and 
Anthony  Morris 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

U.S.  Army  Research  Laboratory 

ATTN:  RDRL-HRM-DI 

Aberdeen  Proving  Ground,  MD  21005-5425 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


3.  DATES  COVERED  (From  -  To) 
October  2010-January  2011 
5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

ARL-TR-5882 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 


11 


Contents 


List  of  Figures  vi 

List  of  Tables  viii 

Executive  Summary  ix 

1.  Introduction  1 

1.1  Background . 1 

1.2  Cargo  Helicopter  -  Engineering  Analysis  Cockpit . 1 

1 .3  CH-47F  Simulator . 1 

1.4  Degraded  Visual  Environment  Simulation . 2 

1.5  Objectives . 2 

2.  Method  2 

2.1  Participants . 2 

2.2  Design,  Training,  and  Mission  Description . 2 

2.3  Displays:  Horizontal  Situation  Display  Hover . 4 

2.3.1  Current  HSDH  Display . 4 

2.3.2  Proposed  HSDH  Display . 5 

2.3.3  Proposed  Decluttered  HSDH  Display . 8 

2.4  Apparatus . 9 

2.4.1  Eye  Tracker  System . 9 

2.4.2  Audio-Video  Collection . 1 1 

2.5  Aircraft  Control  Measures . 11 

2.6  Questionnaires . 11 

2.6. 1  Bedford  Workload  Rating  Scale . 1 1 

2.6.2  Crew  Station  Pilot-Vehicle  Interface . 12 

2.6.3  Training  Assessment . 13 

2.7  Procedure . 13 

2.8  Data  Analysis . 13 

2.9  Limitations . 14 


iii 


3.  Results  14 

3.1  Visual  Gaze . 14 

3.1.1  Fixation  Duration . 15 

3.1.2  Interfixation  Angle . 15 

3.1.3  Fixation  Frequency . 16 

3.1.4  Blink  Frequency . 17 

3.1.5  Horizontal  Situation  Display  Visual  Gaze  Analysis . 17 

3.1.6  Radar  Altimeter  Visual  Gaze  Analysis . 1 8 

3.1.7  Vertical  Speed  Indicator  Visual  Gaze  Analysis . 19 

3.2  Workload . 19 

3.2.1  Overall  Workload  for  HSDH  Display  Type . 19 

3.2.2  Workload  for  Evaluation  Maneuvers  per  HSDH  Display . 20 

3.2.3  Workload  for  Each  Overall  Mission  Task  by  HSDH  Display . 23 

3.3  Crew  Station  Pilot-Vehicle  Interface . 26 

3.3.1  Overall  PVI  Ratings . 26 

3.3.2  PVI  Comparisons  by  HSDH  Display . 26 

3.3.3  Pilot  After  Action  Review  Summary . 29 

3.4  Aircraft  Control  Measures . 30 

3.4.1  Transverse  Landing  Speed . 30 

3.4.2  Vertical  Speed  at  Landing . 30 

3.4.3  Horizontal  Deviation  During  LZ1  Approach . 33 

3.4.4  Horizontal  Deviation  During  Vertical  Descent . 34 

3.4.5  Vertical  Deviation  During  Sling  Load  Operations . 36 

3.5  Training  Assessment . 37 

4.  Discussion  38 

4.1  Visual  Gaze . 38 

4.2  Workload . 40 

4.3  Crew  Station  Pilot-Vehicle  Interface . 40 

4.4  Aircraft  Control  Measures . 41 

4.5  Training  Assessment . 42 

5.  Recommendations  42 

6.  References  44 


iv 


Appendix  A.  Demographic  Questionnaire  45 

Appendix  B.  Bedford  Workload  Rating  Scale  (BWRS)  Questionnaire  49 

Appendix  C.  Pilot  Vehicle  Interface  (PVI)  Questionnaires  53 

Appendix  D.  Summary  of  Pilot  Vehicle  Interface  (PVI)  Comments  71 

Appendix  E.  Pilot  Training  Assessment  Questionnaire  85 

Appendix  F.  Crew  Gaze  Analysis  87 

Appendix  G.  Summary  of  Training  Survey  Comments  91 

Appendix  H.  Summary  of  Bedford  Workload  Rating  Scale  Comments  93 

List  of  Symbols,  Abbreviations,  and  Acronyms  95 

Distribution  List  97 


v 


List  of  Figures 


Figure  1.  Baseline  HSDH . 5 

Figure  2.  Baseline  HSDH  hover  box . 5 

Figure  3.  Proposed  HSDH  display . 6 

Figure  4.  Proposed  HSDH  vertical  speed  tape . 7 

Figure  5.  Proposed  HSDH  target  altitude  for  hover  hangman . 7 

Figure  6.  Proposed  HSDH  horizontal  speed  guidance . 8 

Figure  7.  Proposed  HSDH  vs.  proposed  decluttered  HSDH . 9 

Figure  8.  Eye  tracker,  pupil/camera  monitors,  and  control-panel  interface . 10 

Figure  9.  Pilot  gaze  analysis  scene  planes . 10 

Figure  10.  Average  fixation  duration  across  display  types . 15 

Figure  11.  Average  interfixation  angle  across  display  types . 16 

Figure  12.  Average  fixation  frequency  across  display  types . 16 

Figure  13.  Average  blink  frequency  across  display  types . 17 

Figure  14.  Percentage  of  visual  gaze  on  HSD  AOI . 18 

Figure  15.  Percentage  of  visual  gaze  on  the  RAD  ALT  AOI . 18 

Figure  16.  Percentage  of  visual  gaze  on  the  VSI . 19 

Figure  17.  Crew  overall  workload  ratings  by  HSDH  configuration . 20 

Figure  18.  Baseline  HSDH  mean  workload  ratings  for  mission  task  IDs . 21 

Figure  19.  Proposed  HSDH  mean  workload  ratings  for  mission  task  IDs . 22 

Figure  20.  Proposed  decluttered  HSDH  mean  workload  ratings  for  mission  task  IDs . 22 

Figure  21.  Workload  ratings  for  overall  mission  task  LZ1  approach  landing . 23 

Figure  22.  Workload  ratings  for  overall  mission  task  LZ2  hover  to  land . 24 

Figure  23.  Workload  ratings  for  overall  mission  task  LZ3  sling  load . 24 

Figure  24.  Workload  ratings  for  overall  mission  task  LZ4  high  hover  to  land . 25 

Figure  25.  Workload  ratings  for  overall  mission  task  LZ6  reposition . 26 

Figure  26.  Crew  overall  PVI  ratings  by  HSDH  design . 27 

Figure  27.  PVI  ratings  by  dimension  for  baseline  HSDH . 27 

Figure  28.  PVI  ratings  by  dimension  for  proposed  HSDH . 28 

Figure  29.  PVI  ratings  by  dimension  for  proposed  decluttered  HSDH . 28 

Figure  30.  Transverse  landing  speed  by  pilot . 31 

Figure  31.  Transverse  landing  speed  by  landing  site . 31 


vi 


Figure  32.  Vertical  landing  speed  by  pilot . 32 

Figure  33.  Vertical  landing  speed  by  landing  site . 32 

Figure  34.  Coefficient  of  determination  by  pilot  for  LZ1 . 33 

Figure  35.  Coefficient  of  determination  by  pilot  for  LZ1  with  outliers  removed . 34 

Figure  36.  Horizontal  deviation  from  mean  during  vertical  descent  at  LZ4 . 35 

Figure  37.  Horizontal  deviation  from  desired  LZ4  at  landing . 36 

Figure  38.  Average  altitude  error  for  15-ft  sling  load  operation . 37 

Figure  39.  Overall  training  assessment  ratings  by  question . 37 

Figure  40.  Comparison  of  common  visual  scanning  patterns . 39 

Figure  F-l.  Pilot  visual  gaze  distribution  for  baseline  Horizontal  Situation  Display  Hover 
"  (HSDH) . . . 88 

Figure  F-2.  Pilot  visual  gaze  distribution  for  proposed  HSDH . 89 

Figure  F-3.  Pilot  visual  gaze  distribution  for  decluttered  HSDH . 90 


vii 


List  of  Tables 


Table  1.  Crew  responsibility  and  flight  order . 3 

Table  2.  Mission  tasks  per  evaluation  maneuver . 4 


Executive  Summary 


A  crew  station  working  group  demonstration  was  conducted  on  the  manual  flight  control 
effectiveness  and  usability  of  the  simulated  CH-47F  Horizontal  Situation  Display  Hover  (HSDH) 
pilot  visual  interface.  The  study  compared  three  HSDH  display  configurations:  (1)  the  existing 
CH-47F  baseline  HSDH,  (2)  a  proposed  HSDH  redesign,  and  (3)  a  default  decluttered 
configuration  of  the  proposed  HSDH  redesign.  Eight  career  pilots  evaluated  the  HSDH  usability 
under  simulated  brownout  visual  conditions  for  six  different  hover  and  landing  tasks.  Usability 
was  assessed  along  four  dimensions,  including  cockpit  visual  gaze,  aircraft  control,  pilot 
workload,  and  pilot  vehicle  interface  preference  for  ease  of  use  and  effectiveness  of  information 
presentation.  Crew  gaze  analyses  converged  to  indicate  that  pilots  had  a  collective  reduction  in 
the  amount  of  visual  workload  for  the  proposed  and  decluttered  HSDH  compared  to  the  baseline. 
Aircraft  control  analyses  indicated  that  pilots  had  gains  in  the  overall  aviating  effectiveness 
while  flying  with  the  proposed  and  decluttered  HSDH.  Crews  reported  that  workload  demands 
were  at  acceptable  levels  while  using  the  new  displays  during  all  mission  tasks  included  in  the 
study.  Pilot  feedback  indicated  that  the  interface  readability  of  the  new  displays  was  effectively 
designed  and  presented.  Pilots  highly  recommended  the  overall  design  of  the  proposed  HSDH 
and  voted  unanimously  for  its  immediate  implementation. 


Intentionally  lelt  blank. 


x 


1.  Introduction 


1.1  Background 

Between  October  2001  and  October  2009,  the  greatest  causal  factor  for  rotary  wing  combat  non- 
hostile  airframe  losses  (24%  of  losses)  was  aircraft  operation  in  degraded  visual  environments 
(DVE),  commonly  referred  to  as  brownout  or  whiteout  (Aircraft  Survivability,  2010). 

Brownouts  occur  when  the  helicopter  is  within  ground  effect  and  particulates  are  entrapped  and 
circulated  in  the  rotor  wash,  obscuring  the  view  of  the  surroundings.  In  light  of  this  data, 
developing  Pilot  vehicle  interfaces  that  support  the  safe  and  effective  aviation  of  rotorcraft  in 
poor  visual  conditions  is  a  high  priority. 

The  CH-47F  Chinook  Horizontal  Situation  Display  Hover  (HSDH)  software  crew  station 
working  group  (CSWG)  evaluated  the  design  and  usability  of  hover  flight  management  software 
designed  to  enhance  a  pilot’s  manual  control  of  horizontal  (position,  drift,  and  heading)  and 
vertical  (position  and  speed)  flight  parameters  in  a  brownout  DVE.  The  worst-case  scenario  of 
manual  aircraft  control  without  pilot  access  to  position  hold  and  translation  rate  command  was 
studied.  The  study  compared  three  HSDH  display  configurations:  (1)  the  existing  CH-47F 
baseline  HSDH,  (2)  a  proposed  HSDH  redesign  and  (3)  a  default,  decluttered  configuration  of 
the  proposed  HSDH  redesign. 

1.2  Cargo  Helicopter  -  Engineering  Analysis  Cockpit 

The  Cargo  Helicopter  Program  Management  Office  supports  the  Cargo  Helicopter  -  Engineering 
Analysis  Cockpit  (CH-EAC)  for  designing  and  rapid  prototype  development  and  of  the  Common 
Aviation  Architecture  System  (CAAS).  Pilot-based  studies  and  demonstrations  are  conducted 
using  a  CSWG  method.  The  CSWG  method  is  used  across  multiple  aircraft  platforms  to 
determine  the  best  upgrades  and  modifications  to  the  current  aircraft  (Havir  et  al.,  2006; 

Kennedy  and  Durbin,  2005).  This  method  incorporates  input  from  working  group  members  to 
determine  possible  modifications  which  are  tested  in  a  simulator  using  an  engineering  analysis 
cockpit.  During  this  testing,  performance  is  measured  and  pilots’  observations  and  opinions  are 
collected  to  determine  changes  for  future  design  increments. 

1.3  CH-47F  Simulator 

The  CH-EAC  is  a  CH-47  hull  adapted  to  run  AFS  for  rapid  prototyping  experiments  with 
operational  pilots  in  a  simulated  environment.  The  simulator  is  equipped  with  a  Fokker  Control 
Loader  for  realistic  stick  feedback.  Simulated  flight  characteristics  are  managed  by  an  integrated 
flight  model  from  the  Aviation  Engineering  Directorate  (AED)  for  realistic  flight  performance. 
Pilot  Vehicle  Interface  (PVI)  inputs  for  the  pilots  are  captured  and  time- stamped  for  human 
factors  engineering  (HFE)  analysis.  Audio  is  also  recorded  and  used  for  extra-cockpit 
communication,  such  as  simulated  tower  calls. 
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1.4  Degraded  Visual  Environment  Simulation 


A  physics-based  particle  model  was  used  to  simulate  the  degraded  visual  conditions  of  brownout. 
This  brownout  model  was  incorporated  into  the  out-the-window  system  in  the  Battlefield  Highly 
Immersive  Virtual  Environment  (BHIVE).  The  model  uses  flight  dynamics  from  the  flight  model 
and  helicopter  physical  characteristics  to  generate  a  dynamic  visual  model  of  brownout  conditions. 

1.5  Objectives 

The  primary  objective  of  the  CSWG  demonstration  was  to  determine  whether  the  newly 
designed  HSDH  would  increase  pilot  situational  awareness,  decrease  demands  on  pilot 
workload,  and  decrease  aviating  behaviors  associated  with  brownout  accidents.  Data  were 
obtained  on  crews’  evaluation  and  feedback  for  the  pilot- vehicle  interface,  mission  workload, 
crew  gaze,  and  aircraft  control  measures. 


2.  Method 


2.1  Participants 

Participants  included  eight  male  CH-47  pilots  ranging  in  age  from  25  to  45,  consisting  of  six 
U.S.  Army  active  duty  CH-47  pilots,  one  Department  of  the  Army  Civilian,  and  one  Major  in  the 
Canadian  Air  Force.  Participant  demographics  were  collected  using  the  demographic 
questionnaire  included  in  appendix  A.  The  participants  represented  a  group  of  experienced  pilots 
with  a  U.S.  Army  service  duration  range  of  2-5  years  minimum  and  20-25  years  maximum. 

Total  flight  hours  ranged  from  130  to  631 1  h  (mean  =  3288  h,  median  =  3375).  The  majority 
(69%)  of  their  total  flight  hours  were  in  the  CH-47  (mean  =  2260  h,  median  =  1750),  and  all  but 
one  had  served  as  an  instructor  pilot  (IP)  for  the  CH-47. 

2.2  Design,  Training,  and  Mission  Description 

Participants  were  assigned  into  pilot/co-pilot  crews  of  comparable  rank  and  the  crew  pairings 
were  maintained  for  all  flights  according  to  table  1.  One  exception  to  the  crew  pairing  was  the 
final  crew  in  the  third  mission  set  with  a  decluttered  HSDH.  Pilot  PI  flew  in  the  place  of  pilot 
P6  due  to  pilot  P6  being  reassigned  to  a  different  task.  Each  member  of  a  crew  flew  each 
mission  as  pilot  on  controls  and  as  co-pilot.  As  such,  the  co-pilot  flew  the  next  mission  in  a  set 
as  the  pilot  on  controls.  In  all  flights  except  the  first  crew  of  mission  three  (decluttered  HSDH), 
the  right- seated  pilot  flew  as  first  pilot  on  controls  and  the  left- seated  pilot  flew  as  second  pilot 
on  controls.  This  flight  order  resulted  in  three  opportunities  of  measurement  for  each  pilot  and 
resulted  in  a  repeated-measures  or  within-subjects  design.  Each  crew  flew  a  single  simulated 
mission  in  the  BHIVE.  The  BHIVE  is  developed  and  maintained  by  the  System  Simulation  and 
Development  Directorate  (SSDD)  of  the  U.S.  Army  Aviation  and  Missile  Research, 
Development,  and  Engineering  Command  (AMRDEC)  at  Redstone  Arsenal,  AL. 
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Table  1.  Crew  responsibility  and  flight  order. 


Pilot 

Baseline 

Proposed 

Decluttered 

1st 

P7,  P2,  P5,  P6 

P7,  P2,  P5,  P6 

P8,  P2,  P5,  P3 

2nd 

P8.P4.P1.P3 

P8.  P4,  P1.P3 

P7,  P4,  P6,  PI 

Training  on  the  CH-EAC  simulator  for  the  CH-47F  was  conducted  for  all  pilots  on  the  first  day 
of  test  and  before  the  simulated  flights  of  record.  In  addition,  specific  training  for  the  current 
HSDH  and  its  proposed  modifications  was  conducted  prior  to  flights  of  record  on  the  simulated 
mission.  Before  entering  the  simulation,  crews  were  provided  a  mission  brief  and  their  flight 
plan. 

The  flight  mission  was  flown  in  and  around  simulated  terrain  from  Fort  Irwin  and  National 
Training  Center  (NTC).  The  mission  was  executed  under  visual  flight  rules  (VFR)  conditions. 
The  flight  plan  included  six  evaluation  maneuvers  that  punctuated  the  en-route  flight  legs  of  the 
mission.  The  evaluation  maneuvers  were  constructed  using  guidance  from  the  Aircrew  Training 
Manual  (ATM)  Cargo  Helicopter,  CH-47D/F  tasks  1039  Perform  Hovering  Flight  Utilizing 
Symbology,  1058  Perform  Visual  Meteorological  Conditions  Approach,  and  1063  Perform 
External  Foad  Operations.  The  mission  was  designed  to  have  multiple  types  of  landing  and 
hover  events.  Each  of  the  six  maneuvers  was  designed  so  as  to  exercise  pilots’  manual  control  of 
the  aircraft  during  brownout  DVE.  Manual  control  of  the  aircraft  was  implemented  in  the 
simulator  by  turning  off  the  position  hold  (P-Hold)  and  translation  rate  command  (TRC) 
capability.  The  DVE  condition  existed  at  40  ft  and  below.  The  DVE  conditions  reinforced  the 
situational  demand  for  pilots  to  rely  on  the  HSDH  during  the  maneuvers.  The  mission 
background  events  included  standard  personnel/equipment  air  transport.  The  missions  were 
developed  by  subject  matter  experts  (SMEs)  within  SSDD  at  the  AMRDEC. 

The  conditions  of  these  landing  zones  were  designed  to  require  different  levels  of  control  and 
enforce  different  types  of  evaluation  maneuvers.  Landing  Zone  1  (LZ1)  was  in  an  open  area 
with  a  smoke  plume  marking  the  desired  landing  site.  No  hover  was  required,  and  an  alpha 
approach  landing  was  used.  Landing  Zone  2  (LZ2)  was  located  in  a  courtyard  next  to  a  mosque. 
The  approach  to  LZ2  was  relatively  open  with  a  few  obstacles  and  vehicles  located  in  the  landing 
area.  There  was  a  20-ft  hover  reference  point  defined  directly  above  the  landing  zone.  During 
landing  Zone  3  (LZ3),  a  sling  load  operation  was  performed.  There  was  a  50-ft  hover  reference 
point  horizontally  offset  from  a  15 -ft  hover  reference  point  above  the  load.  The  aircraft  was  not 
to  land  at  this  location.  Landing  Zone  4  (LZ4)  was  located  next  to  a  tall  building  and  trees.  The 
configuration  of  this  location  required  a  high  hover  with  a  vertical  descent.  There  was  a  hover 
reference  point  located  80  ft  above  the  landing  zone.  Bicycle  Lake  Army  Airfield  (KBYS)  was 
the  last  location  in  the  flight  plan.  This  landing  zone  was  in  an  open  area.  No  hover  was 
required,  and  an  alpha  approach  landing  was  used.  A  final  reposition  was  requested  after  the 
initial  landing  at  KBYS.  This  was  not  a  planned  location,  but  the  crew  was  simply  told  to 
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reposition  the  aircraft  a  certain  distance  away  in  a  cardinal  direction.  The  pilot  was  free  to 
choose  to  land  the  aircraft  in  a  variety  of  manners.  The  full  list  of  mission  tasks  for  each  landing 
zone  is  presented  in  table  2. 

Table  2.  Mission  tasks  per  evaluation  maneuver. 


LZ1  Alpha 
Landing 

LZ2  Hover  to 
Landing 

LZ3  Sling 
Load 

LZ4  Hover  to 
Landing 

LZ5  Alpha 
Landing  Bike 
Lake 

LZ6 

Reposition 

1.  LZ1  overall 

6.  LZ2  overall 

9.  LZ3  overall 

12.  LZ4  overall 

1.  Maintain 

15.  Perform  a 

approach  to 

controlled 

2.  Maintain 

7.  Perform  a 

10.  Perform  a 

13.  Perform  a 

desired  landing 

termination  to 

approach  to 

controlled 

controlled 

controlled 

point 

initial  hover 

desired  landing 

termination  to 

termination  to 

termination  to 

point 

initial  hover 

initial  hover 

initial  hover 

2.  Maintain 

rate  of  closure 

3.  Maintain  rate 

8.  Perform  a 

1 1 .  Perform  a 

14.  Perform  a 

of  closure 

controlled  descent 

controlled 

controlled  descent 

3.  Maintain 

with  minimal  drift 

termination  to 

with  minimal  drift 

ground  track 

4.  Maintain 

hookup  point 

alignment 

ground  track 

alignment 

4.  Perform  a 

controlled 

5.  Perform  a 

termination  to 

controlled 

touch-down 

termination  to 

touchdown 

2.3  Displays:  Horizontal  Situation  Display  Hover 
2.3.1  Current  HSDH  Display 

The  CH-47  HSDH  baseline  symbology  shown  in  figure  1  includes  multiple  indicators  which  are 
used  in  concert  to  control  the  vertical  position  and  velocity  of  the  aircraft.  Of  particular  interest 
to  the  CSWG  Demonstration  were  the  vertical  speed  indicator  (VSI),  the  radar  altitude  (RAD 
ALT)  indicator,  and  the  hover  box.  The  VSI  displays  vertical  speed  from  -1000  ft/min  (fpm)  to 
1000  fpm.  The  RAD  ALT  indicator  displays  the  altitude  above  the  terrain  presently  beneath  the 
aircraft  from  0  to  525  ft.  The  hover  box  provides  visual  reference  of  error  between  the  reference 
altitude  and  the  radar  altitude  by  increasing  in  size  as  the  aircraft  descends,  and  decreasing  in 
size  when  the  aircraft  ascends.  The  scaling  of  the  hover  box  can  be  seen  in  figure  2  and  can  be 
used  by  the  pilot  for  control  around  the  desired  altitude  that  has  been  entered  into  the  control 
display  unit  (CDU). 
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Hover  Box 


Radar  Altitude 


Vertical  Speed 
Indicator 


Figure  1.  Baseline  HSDH. 
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Figure  2.  Baseline  HSDH  hover  box. 


2.3.2  Proposed  HSDH  Display 

The  proposed  HSDH,  shown  in  figure  3,  incorporates  changes  to  the  baseline  symbology  to  be 
used  during  hover  and  landing.  The  vertical  speed  tape,  which  is  used  to  control  the  vertical 
speed  and  position  of  the  aircraft,  was  added.  This  tape  has  markings  on  a  dual-purpose  scale 
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Figure  3.  Proposed  HSDH  display. 

with  marks  at  100  fpm,  per  tick  mark,  for  indicating  vertical  speed,  and  20  ft  per  tick  mark  for 
indicating  radar  altitude.  The  dual-purpose  scale  makes  available  to  the  pilot  the  relative 
information  between  the  rate  of  descent  and  the  location  of  the  ground  plane.  This  proposed 
design  makes  available  two  pieces  of  information  for  one  detection  effort.  The  vertical  speed 
tape  has  a  bar  that  extends  up  or  down  from  the  center  horizontal  line  based  on  the  current 
vertical  speed  of  the  aircraft.  The  color  of  this  bar  changed  based  on  various  conditions,  as 
shown  in  figure  4. 
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Open  Green  = 
Vertical  Speed 
Tape  is  Not  Past 
the  Top  of  the 
Rising  Ground 


Solid  Green  = 
Vertical  Speed 
Tape  is  Past 
the  Rising  Ground 


Solid  Yellow  = 
Vertical  Speed 
Tape  is  greater 
than  the  landing 
limit 


Figure  4.  Proposed  HSDH  vertical  speed  tape. 

The  vertical  speed  tape  also  shows  a  representation  of  the  distance  to  ground  as  measured  by  the 
radar  altimeter.  This  ground  indicator  will  first  appear  at  100  ft  above  the  ground  at  the  bottom 
of  the  tape  and  will  move  upward  toward  the  center  horizontal  line  as  the  aircraft  approaches  the 
ground,  becoming  even  with  the  center  horizontal  line  when  the  RAD  ALT  is  at  0.  This 
indicator  can  also  be  used  to  show  a  reference  altitude  if  entered  into  the  CDU  or  flight  plan. 

The  height  of  the  reference  altitude  is  represented  by  the  top  line  of  the  hangman,  as  seen  in 
figure  5. 


Figure  5.  Proposed  HSDH  target  altitude  for  hover  hangman. 
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For  controlling  the  aircraft  in  the  horizontal  direction,  the  HSDH  has  an  acceleration  cue,  which 
is  represented  by  the  green  circle  (see  figure  6)  and  shows  the  acceleration  vector  of  the  aircraft, 
and  a  velocity  vector,  which  is  represented  by  the  green  line  extending  from  the  center  of  the 
HSDH  and  shows  the  direction  and  speed  of  the  aircraft.  The  proposed  HSDH  added  a 
horizontal  speed  guidance  cue,  which  gives  the  pilot  an  indication  of  where  the  acceleration  cue 
should  be  placed  in  order  to  direct  the  aircraft  to  the  desired  hover  or  landing  point.  This  is 
represented  by  the  magenta  horseshoe  as  shown  in  figure  6.  This  horseshoe  changed  into  a  white 
circle  when  the  desired  acceleration  is  zero,  indicating  that  the  aircraft  is  hovering  at  the  desired 
point.  Also,  a  target  heading  indicator  was  added  to  HSDH,  as  shown  in  figure  3. 


Figure  6.  Proposed  HSDH  horizontal  speed  guidance. 

2.3.3  Proposed  Decluttered  HSDH  Display 

The  proposed  HSDH  default  decluttered  configuration  consists  of  several  changes  to  simplify  the 
look  of  the  HSDH  and  eliminate  unnecessary  information  as  seen  in  figure  7.  The  decluttered 
HSDH  removes  the  torque  tape  and  replaces  it  with  simply  the  torque  digital  readout.  The 
decluttered  HSDH  also  removes  the  air  speed  tape.  Both  of  these  tapes  are  still  available  to  the 
pilot  on  the  vertical  situation  display  (VSD),  located  on  the  upper  half  of  the  multi-function 
display  (MFD).  The  ground  speed  is  moved  from  the  bottom  of  the  vertical  speed  tape  to  the  left 
side  of  the  center  horizontal  line  to  bring  it  closer  to  the  other  important  gauges  used  when  hover 
or  landing. 
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Figure  7.  Proposed  HSDH  vs.  proposed  decluttered  HSDH. 

2.4  Apparatus 

2.4.1  Eye  Tracker  System 

Crew  visual  gaze  and  dwell  times  were  collected  with  a  head  and  eye  tracking  system  from 
Applied  Science  Laboratories  (ASL)  depicted  in  figure  8.  This  system  was  selected  because  of 
its  eye-head  integration  capability  that  allowed  for  unrestricted  head  movement.  The  ASL 
eye-head  package  included  a  Model  H6  eye  tracker  and  NDI  Polaris  optical  head  tracker.  The 
ASL  software  allowed  continuous  monitoring  and  data  collection  of  crew  (pilot  and  co-pilot)  eye 
positions.  This  technology  allowed  data  collection  for  predefined  functional  areas  of  the  CH-47 
CHE  AC  cockpit. 

Crew  gaze  analysis  determined  the  total  gaze  time  in  each  of  the  three  different  areas  of  interest 
(AOI)  for  the  pilot,  shown  in  figure  9.  The  three  AOIs  were  defined  for  the  outboard  MFD  at 
each  crew  station:  RAD  ALT,  VSI,  and  horizontal  situation  display  (HSD).  The  eye  gaze 
analysis  results  are  presented  in  section  3.1. 
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Figure  8.  Eye  tracker,  pupil/camera  monitors,  and  control-panel  interface. 


Figure  9.  Pilot  gaze  analysis  scene  planes. 
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2.4.2  Audio-Video  Collection 

The  BHIVE  includes  a  Battlemaster,  an  exercise  control  station  that  has  access  to  the  currently 
running  simulation.  The  Battlemaster  station  has  access  to  data  collection  devices,  headset 
communications,  and  video  monitoring.  Audio  and  video  feeds  were  recorded  for  crew 
communications,  as  well  as  aircraft  telemetry  and  pilot- vehicle  interface  displays.  The 
Battlemaster  station  provided  the  SMEs  with  the  information  needed  to  coordinate  the  scenario- 
driven  events  and  data  collection  devices  required  for  the  CSWG  events. 

2.5  Aircraft  Control  Measures 

Information  on  the  aircraft’s  position  and  velocity  was  recorded  directly  from  the  simulator 
throughout  all  runs.  This  data  was  then  analyzed  to  determine  the  controllability  of  various 
aspects  of  the  aircraft’s  flight.  The  various  analyses  conducted  were  transverse  landing  speed, 
vertical  speed  at  landing,  linearity  of  approach  to  LZ1,  verticalness  of  approach  to  LZ4,  accuracy 
of  landing  at  LZ4,  and  vertical  deviation  during  sling  load  operations.  The  transverse  landing 
speed  provides  a  measure  of  the  danger  of  a  rollover  on  landing,  with  higher  values  indicating  a 
more  dangerous  landing.  The  vertical  speed  at  landing  was  measured  to  indicate  whether  or  not 
the  various  landings  were  at  rates  higher  than  the  maximum  rate  of  descent  at  landing,  which 
could  possibly  cause  damage  to  the  aircraft.  The  linearity  of  approach  to  LZ1  and  the 
verticalness  of  approach  to  LZ4  were  intended  to  measure  the  horizontal  controllability  of  the 
aircraft  during  these  maneuvers.  The  vertical  deviation  during  sling  load  operations  was 
intended  to  measure  the  vertical  controllability  of  the  aircraft  during  hover  maneuvers.  The 
accuracy  of  landing  at  LZ4  was  measured  to  indicate  how  close  the  aircraft  was  being  landed  to 
the  desired  landing  location. 

2.6  Questionnaires 

2.6.1  Bedford  Workload  Rating  Scale 

To  estimate  the  level  of  spare  workload  capacity  needed  to  perform  the  landing  and  hover  tasks 
under  brownout  DVE  conditions,  the  participants  completed  the  Bedford  Workload  Rating  Scale 
(BWRS)  (appendix  B)  immediately  after  each  mission.  Participants  completed  the  BWRS  to  rate 
the  workload  needed  to  accomplish  each  major  task  in  the  mission  (refer  to  table  2). 

The  BWRS  has  been  used  extensively  by  the  military,  civil,  and  commercial  aviation 
communities  for  pilot  workload  estimation  (Roscoe  and  Ellis,  1990).  It  requires  pilots  to  rate  the 
level  of  workload  associated  with  a  task  based  on  the  amount  of  spare  workload  capacity  they 
estimate  they  have  to  perform  additional  tasks.  Spare  workload  capacity  is  an  important 
commodity  for  pilots  because  they  are  often  required  to  perform  several  tasks  concurrently.  For 
example,  co-pilots  often  perform  navigational  tasks,  communicate  via  multiple  radios,  monitor 
aircraft  systems,  and  assist  the  pilot  with  controls  of  flight  tasks  (e.g.,  maintain  air  space 
surveillance)  within  the  same  time  interval.  Mission  performance  is  reduced  if  pilots  are  task 
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saturated  and  have  little  or  no  spare  capacity  to  perform  other  tasks.  Design  elements  of  the 
CH-47F  HSDH  pages  should  ensure  that  pilots  can  maintain  adequate  spare  workload  capacity 
while  performing  flight  and  mission  tasks.  The  BWRS  results  are  presented  in  section  3.2. 

2.6.2  Crew  Station  Pilot-Vehicle  Interface 

The  design  of  a  crew  station  PVI  influences  crew  workload  and  situational  awareness  (SA) 
during  a  flight  mission.  A  crew  station  that  is  designed  to  augment  the  cognitive  and  physical 
abilities  of  crews  will  minimize  workload,  enhance  SA,  and  contribute  to  successful  mission 
performance.  To  assess  the  PVI,  the  crews  completed  a  comprehensive  questionnaire 
(appendix  C)  at  the  end  of  each  mission  for  the  different  HSDH  implementations.  The  PVI 
questionnaire  addressed  usability  and  informational  effectiveness  of  the  different  HSDH 
configurations:  baseline,  proposed,  and  proposed  default  decluttered. 

All  question  items  in  the  PVI  questionnaire,  except  PV-5,  were  rated  on  a  4-point  Likert-type 
scale.  Questions  in  group  PV-1  had  anchors  of  very  effective,  somewhat  effective,  somewhat 
in-effective,  and  very  in-effective  (scored  1 — 4,  respectively).  Questions  in  the  PVI-2  group  had 
anchors  of  excellent,  good,  poor,  and  unsatisfactory  (scored  1^1,  respectively).  Questions  in  the 
PVI-3  group  had  anchors  of  very  easily,  somewhat  easily,  somewhat  difficult,  and  very  difficult 
(scored  1-4,  respectively).  Questions  in  group  PV-4  had  anchors  of  very  effective,  somewhat 
effective,  somewhat  in-effective,  and  very  in-effective  (scored  1 — 4,  respectively).  Therefore, 
lower  scores  indicate  positive  crew  station  usability.  Response  options  for  questions  in  the  PV-5 
group  were  open-ended,  written  reports.  Pilot  reports  are  presented  in  appendix  D,  “Summary  of 
PVI  Comments.”  Participants  were  also  solicited  to  list  in  open  response  format  any  comment 
on  the  PVI  dimensions  queried  on  the  design  of  the  different  HSDH  configurations.  The  PVI 
results  are  presented  in  section  3.3. 

The  baseline  HSDH  PVI  questionnaire  consisted  of  15  question  items,  in  four  main  categories: 
eight  questions  addressed  the  effectiveness  of  HSDH  vehicle  interface  cues  for  supporting  the 
mission  evaluation  tasks  (PV-1),  four  questions  addressed  the  informational  value  of  the  HSDH 
symbols  and  cues  (PV-2),  one  question  assessed  the  ease  of  understanding  the  color  schemes  on 
the  HSDH  screen  (PV-3),  and  two  questions  addressed  the  overall  effectiveness  of  the  HSDH  to 
support  the  mission  and  pilot  situational  awareness  (PV-4).  The  final  question  (PV-5)  allowed 
pilots  to  write  in  comments  about  the  likes  and  dislikes  of  the  baseline  HSDH. 

The  proposed  HSDH  PVI  questionnaire  consisted  of  16  question  items,  in  four  main  categories: 
eight  questions  addressed  the  effectiveness  of  HSDH  vehicle  interface  cues  for  supporting  the 
mission  evaluation  tasks  (PV-1),  five  questions  addressed  the  informational  value  of  the  HSDH 
symbols  and  cues  (PV-2),  one  question  assessed  the  ease  of  understanding  the  color  schemes  on 
the  HSDH  screen  (PV-3),  and  two  questions  addressed  the  comparison  of  the  baseline  and  the 
proposed  HSDH  displays  for  overall  effectiveness  in  supporting  the  mission  and  pilot  situational 
awareness  (PV-4).  The  final  question  (PV-5)  allowed  pilots  to  write  in  comments  about  the  likes 
and  dislikes  of  the  proposed  HSDH. 
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The  proposed  decluttered  HSDH  PVI  questionnaire  consisted  of  20  question  items,  in  four  main 
categories:  eight  questions  addressed  the  effectiveness  of  HSDH  vehicle  interface  cues  for 
supporting  the  mission  evaluation  tasks  (PV-1),  seven  questions  addressed  the  informational 
value  of  the  HSDH  symbols  and  cues  (PV-2),  one  question  assessed  the  ease  of  understanding 
the  color  schemes  on  the  HSDH  screen  (PV-3),  and  four  questions  addressed  the  comparisons 
among  the  baseline,  proposed  and  decluttered  HSDH  displays  for  overall  effectiveness  in 
supporting  the  mission  and  pilot  situational  awareness  (PV-4).  The  final  question  (PV-5) 
allowed  pilots  to  write  in  comments  about  the  likes  and  dislikes  of  the  proposed  decluttered 
HSDH. 

2.6.3  Training  Assessment 

Participants  completed  a  training  assessment  questionnaire  (appendix  E)  after  the  simulator  and 
HSDH  display  training.  The  training  assessment  questionnaire  consisted  of  five  question  items, 
rated  on  a  4-point  Likert-type  scale  with  anchors  of  strongly  disagree,  disagree,  agree,  and 
strongly  agree  (scored  1-4,  respectively).  Therefore,  higher  scores  indicate  positive  attitudes 
toward  the  training.  Participants  were  also  asked  to  list  three  positive  aspects  of  the  training  and 
three  improvements  that  could  be  made  for  training.  The  training  assessment  results  are 
presented  in  section  3.5. 

2.7  Procedure 

Participants  received  one  day  of  training  on  the  CHEAC  simulator  and  the  three  software 
implementations  of  the  baseline,  proposed,  and  decluttered  HSDH  displays.  Flight  procedures 
with  relevant  mission  scenario  problems  were  operationally  reviewed,  and  software  page 
sequences  were  trained  in  a  series  of  increasing  fidelity  sessions  which  include  classroom  lecture 
format,  desktop  software  trainer,  and  CHEAC  simulation  flights.  All  participants  were  allowed 
free  discussion  to  ask  questions  throughout  the  day  of  training.  At  the  end  of  the  training  day, 
participants  completed  the  training  assessment  questionnaire.  Also  at  the  end  of  the  training  day, 
crew  1  flew  the  first  mission.  All  crews  consecutively  flew  the  mission  using  the  baseline 
HSDH,  the  proposed  HSDH,  and  finally  the  decluttered  HSDH.  Before  the  start  of  each  flight, 
the  pilot  and  co-pilot  were  fitted  with  the  ASL  eye  tracker  and  calibrated.  After  each  pilot  of  a 
crew  pair  flew  the  mission  to  completion,  participants  immediately  completed  the  BWRS  and 
PVI  questionnaires.  An  after-action  review  (AAR)  was  conducted  for  each  crew  individually. 
Upon  completing  flights  utilizing  the  three  HSDH  designs,  an  overall  AAR  was  conducted  with 
all  pilots  present,  except  P6,  who  was  reassigned  to  another  location  after  the  second  flight  mission. 

2.8  Data  Analysis 

Descriptive  statistics  were  calculated  for  responses  to  the  training  assessment,  BWRS,  and  PVI 
questionnaires.  Where  shown  in  the  figures,  error  bars  indicate  the  95%  confidence  interval  (Cl) 
of  the  mean.  These  error  bars  can  be  used  to  help  make  conclusions  about  statistically 
significant  differences.  If  two  95%  Cl  error  bars  do  not  overlap  and  the  sample  sizes  are  nearly 
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equal,  the  difference  is  statistically  significant  with  a  P  value  much  less  than  0.05  (Payton  et  al., 
2003).  To  confirm  statistically  significant  differences,  analysis  of  variance  (ANOVA)  and/or 
t-tests  was  conducted. 

2.9  Limitations 

Some  limitations  should  be  considered  when  interpreting  the  data  presented  in  the  results 
section: 

•  There  was  a  small  sample  size  of  crew  members  ( N  =  8)  that  may  not  be  representative  of 
the  total  pilot  population. 

•  Pilot  P6  left  the  study  early  missing  his  last  flight. 

•  The  small  sample  size  limits  the  power  of  the  statistical  analyses  to  find  true  effects. 

•  The  small  sample  size  limits  the  confidence  in  generalizing  results  to  a  broader  pilot 
population. 

•  Training  scope  and  duration  were  short.  This  may  have  limited  the  pilot’s  mastery  of  the 
simulator  and  contributed  to  learning  effects  and  order  effects  (i.e.,  display  presentation 
was  not  counterbalanced)  during  the  flights  of  record. 

•  Flights  were  conducted  in  a  simulator.  Although  the  simulator  has  a  high  degree  of 
fidelity,  some  discrepancies  between  the  simulator  and  actual  aircraft  performance  may 
exist. 

These  limitations  are  not  uncommon  when  replicating  a  complex  aviation  system  in  a  simulator. 
However,  the  data  and  the  information  given  in  the  Results  and  Summary  sections  of  this  report 
should  be  interpreted  based  on  these  limitations.  Additional  data  should  be  collected  during 
future  simulations  and  tests  to  augment  and  expand  the  findings  contained  in  this  report. 


3.  Results 


3.1  Visual  Gaze 

Visual  gaze  behavior  was  analyzed  using  four  ocular  activity  metrics  and  localized  areas  of 
interest  (AOI)  analyses.  Specifically,  fixation  duration,  interfixation  angle,  fixation  frequency, 
and  blink  frequency  were  analyzed  as  behavioral  measures,  and  the  HSD,  VSI,  and  RAD  ALT 
AOIs  were  taken  as  measures  of  visual  attention.  Statistical  comparisons  (one-way  repeated 
measures  ANOVA)  were  conducted  for  each  of  these  measures  with  display  type  (baseline 
HSDH,  proposed  HSDH,  and  decluttered  HSDH)  as  the  independent  variable. 
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3.1.1  Fixation  Duration 

In  order  to  measure  the  cognitive  cost  of  processing  the  proposed  HSDH  symbology  compared  to 
the  baseline  symbology,  fixation  duration  data  were  analyzed  and  statistically  compared  across 
different  display  types  while  collapsing  data  across  a  variety  of  hover  tasks.  Fixation  duration 
indicates  the  length  of  time  a  pilot’s  visual  attention  is  captured  by  a  given  stimulus.  Often,  this 
is  taken  as  a  measure  of  mental  workload  (Tole  et  al.,  1983),  in  which  longer  fixation  durations 
indicate  greater  workload  because  of  longer  processing  cycles. 

Results  can  be  viewed  in  figure  10,  which  indicate  no  statistical  difference,  F(2,  6)  =  1.536, 
p  >  0.05  across  display  conditions. 
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Figure  10.  Average  fixation  duration  across  display  types. 

3.1.2  Interfixation  Angle 

In  order  to  investigate  the  amount  of  effort  exerted  by  the  visual  system  as  a  result  of  changing 
the  HSDH  symbology,  interfixation  angle,  a  different  measure  of  visual  workload  was  compared 
across  display  types  while  collapsing  data  across  all  of  the  hover  conditions.  Interfixation  angle, 
as  measured  in  this  analysis,  is  the  angular  distance  between  gaze  fixations.  A  reduction  in 
interfixation  angle  indicates  that  less  effort  was  required  by  the  visual  system  to  obtain 
information  related  to  the  pilot’s  task. 

A  one-way  repeated  measures  ANOVA  indicated  a  strong  trend  towards  significance,  F( 2,  6)  = 
5.513,p  =  0.057,  suggesting  that  the  pilots’  visual  system  effort  requirements  were  sequentially 
reduced  with  the  introduction  of  the  proposed  and  decluttered  HSDH  symbology.  These  data  are 
graphically  represented  in  figure  11. 
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Figure  11.  Average  interfixation  angle  across  display  types. 

3.1.3  Fixation  Frequency 

One  purpose  of  the  proposed  HSDH  symbology  was  to  localize  the  pilot’s  visual  attention  to  a 
more  central  position  when  monitoring  information  related  to  the  pilot’s  hover  position.  If  this 
were  achieved,  then  the  frequency  of  fixations  required  to  gather  this  data  should  have  decreased, 
indicating  that  pertinent  information  for  the  pilot  was  more  readily  available.  Essentially,  it 
means  that  the  “units  of  information”  per  fixation  would  increase,  thereby  requiring  fewer 
fixations  to  achieve  the  same  job. 

Data  can  be  viewed  in  figure  12,  which  indicates  that  there  were  no  significant  differences  in 
average  fixation  frequency  across  display  conditions,  F( 2,  6)  =  0.226,  p  >  0.05. 
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Figure  12.  Average  fixation  frequency  across  display  types. 
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3.1.4  Blink  Frequency 


Blink  frequency  is  associated  with  both  visual  and  mental  workload,  and  some  previous  research 
has  demonstrated  a  negative  correlation  between  these  variables  (Veltman  and  Gaillard,  1998). 
As  blinks  increase  workload  tends  to  be  reduced.  The  following  analysis  evaluates  blink 
behavior  as  a  function  of  display  type. 


A  one-way  repeated  measures  ANOVA  indicates  that  blink  behavior  was  significantly  inhibited 
as  a  function  of  display  type,  F( 2,  6)  =  9.753,  p  =  0.021.  Figure  13  graphically  presents  these 
data. 


Average  Blink  Frequency 


0.09 
0.08 
0.07 
8  0.06 

J2  0.05 

c 

to  0.04 
0.03 
0.02 
0.01 


Baseline  Proposed  Decluttered 


Figure  13.  Average  blink  frequency  across  display  types. 

3.1.5  Horizontal  Situation  Display  Visual  Gaze  Analysis 

The  following  HSD  analysis  was  conducted  to  investigate  if  the  pilot’s  visual  attention  was  more 
consistently  allocated  to  the  HSD  symbology  as  a  function  of  the  display  type.  The  purpose  of 
the  new  symbology  was  to  enhance,  and  consequently  reposition,  the  same  information  displayed 
by  the  RAD  ALT  and  VSI.  The  enhanced  symbology  was  placed  within  the  AOI  of  the  HSD. 
Consequently,  if  the  enhanced  HSD  symbology  successfully  provided  the  required  RAD  ALT 
and  VSI  information  to  the  pilot,  then  the  visual  gaze  percentages  within  the  HSD  symbology 
should  be  reduced  from  the  RAD  ALT  and  VSI  and  should  increase  within  the  HSD  AOI.  The 
following  statistical  analysis  evaluates  the  varying  gaze  percentages  of  the  HSD  across  display 
types. 

Figure  14  presents  a  significant  increase  in  the  percentage  of  time  spent  visually  gazing  over  the 
HSD  AOI,  F( 2,  6)  =  6.782,  p  =  0.04.  This  indicates  that  the  pilots  were  able  to  continuously  rely 
on  the  enhanced  symbology  for  the  required  hover  information. 


17 


Percentage  of  Visual  Gaze  in  HSD 

90.6 


Baseline _ Proposed _ Decluttered 


Figure  14.  Percentage  of  visual  gaze  on  HSD  AOI. 


3.1.6  Radar  Altimeter  Visual  Gaze  Analysis 

Similar  to  the  previous  analysis,  the  percentage  of  visual  gaze  time  spent  over  the  RAD  ALT  was 
analyzed  in  order  to  determine  whether  pilots  relied  on  the  hangman  symbology  for  information 
that  is  traditionally  presented  on  the  RAD  ALT.  This  analysis  determines  an  objective  pilot 
preference  for  each  information  source. 


Figure  15  illustrates  that  there  is  no  significant  difference  in  visual  gaze  on  the  RAD  ALT 
exhibited  across  display  type,  F( 2,  6)  =  0.222,  p  >  0.05.  These  results  indicate  that,  although  there 
was  a  significant  increase  in  the  visual  gaze  of  the  HSD  symbology,  pilots  still  relied  just  as  heavily 
on  the  RAD  ALT  in  terms  of  total  time  spent  visually  dwelling  on  the  RAD  ALT  symbology. 
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Figure  15.  Percentage  of  visual  gaze  on  the  RAD  ALT  AOI. 
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3.1.7  Vertical  Speed  Indicator  Visual  Gaze  Analysis 

Similar  to  the  previous  analysis,  the  percentage  of  visual  gaze  time  spent  over  the  VSI  was 
analyzed  in  order  to  determine  if  pilot’s  relied  on  the  vertical  speed  tape  symbology  for 
information  that  is  traditionally  presented  on  the  VSI.  This  analysis  determines  an  objective 
distribution  of  a  pilot’s  focus  of  attention. 

Figure  16  illustrates  a  significant  difference  in  visual  gaze  on  the  VSI  was  exhibited  across 
display  type,  F( 2,  6)  =  4.05,  p  =  0.045.  These  results  indicate  that  there  was  a  significant 
decrease  in  the  visual  gaze  on  the  VSI,  indicating  pilots  used  the  vertical  speed  tape  for 
information  that  is  usually  presented  on  the  VSI  in  the  baseline  HSDH. 
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Figure  16.  Percentage  of  visual  gaze  on  the  VSI. 

3.2  Workload 

The  BWRS  was  constructed  to  assess  a  pilot’s  subjective  perception  of  available  spare  mental 
workload  during  the  evaluation  maneuvers  performed  at  each  mission  task  site.  BWRS  scores 
were  taken  for  each  landing  zone  overall  and  for  the  ATM  subtasks  that  constituted  the  mission 
at  a  particular  flight  plan  waypoint.  A  workload  rating  between  2  and  3  indicates  that  the  rater 
had  “enough  workload  capacity  for  all  desirable  additional  tasks.”  BWRS  scores  greater  than  6 
are  not  considered  “tolerable  for  the  task”  and  therefore  are  considered  as  high  workload. 

3.2.1  Overall  Workload  for  HSDH  Display  Type 

The  overall  pilot  BWRS  score  for  all  mission  task  IDs  (see  table  2)  were  3.58,  3.44,  and  3.06  for 
baseline  HSDH,  proposed  HSDH,  and  decluttered  HSDH,  respectively  (see  figure  17).  Error 
bars  represent  the  95%  confidence  interval  of  the  mean.  The  combined  BWRS  scores  across 
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evaluation  tasks  graphically  appear  statistically  similar,  as  indicated  by  overlap  of  the  95% 
confidence  interval  error  bars.  A  one-way,  repeated  measures  ANOVA  also  indicated  no 
significant  statistical  difference  among  HSDH  configurations  F  (2,  14)  =  0.64,  p  >  0.05.  Data 
indicate  that  pilots  reported  having  adequate  spare  workload  capacity  while  utilizing  the  three 
HSDH  displays. 


Figure  17.  Crew  overall  workload  ratings  by  HSDH  configuration. 

3.2.2  Workload  for  Evaluation  Maneuvers  per  HSDH  Display 

The  BWRS  results  for  the  mission  task  IDs  (see  table  2)  performed  with  the  baseline  HSDH  are 
presented  in  figure  18.  Although  the  maximum  mean  BWRS  score  is  4.5  for  Mission  Task  ID  6, 
LZ2  hover  to  landing,  one  pilot  indicated  a  BWRS  score  of  7  on  performing  three  tasks  with  the 
baseline  HSDH.  The  tasks  thus  rated  were:  performing  a  controlled  termination  to  touchdown  at 
LZ1,  performing  a  controlled  descent  with  minimal  drift  at  LZ2,  and  performing  a  controlled 
descent  with  minimal  drift  at  LZ4.  Even  though  some  of  the  evaluation  maneuvers  were 
challenging,  pilots  reported  overall  having  enough  spare  workload  capacity  for  easy  additional 
tasks. 
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Figure  18.  Baseline  HSDH  mean  workload  ratings  for  mission  task  IDs. 

The  BWRS  results  for  the  mission  task  IDs  (see  table  2)  performed  with  the  proposed  HSDH  are 
presented  in  figure  19.  Although  the  maximum  mean  BWRS  score  is  4.13  for  Mission  Task  ID 
11,  LZ3  Sling  load,  one  pilot  indicated  a  BWRS  score  of  7  while  performing  task  ID  eight  with 
the  proposed  HSDH.  The  tasks  rated  greater  than  6  on  BWRS  were:  LZ2  overall  hover  to 
landing  (ID  6),  performing  a  controlled  termination  to  touchdown  at  LZ2  (ID  8),  LZ3  sling  load 
overall  (ID  9),  performing  a  controlled  termination  to  initial  hover  at  LZ3  (ID  10),  perform  a 
controlled  termination  to  hookup  point  at  LZ3  (ID  1 1),  LZ4  overall  hover  to  landing  (ID  12), 
performing  a  controlled  termination  to  initial  hover  at  LZ4  (ID  13),  and  performing  a  controlled 
descent  with  minimal  drift  at  LZ4  (ID  14).  Again,  the  hover  maneuvers  were  challenging,  but 
pilots  reported  having  additional  spare  workload  capacity  for  some  attention  to  additional  tasks. 

The  BWRS  results  for  the  mission  task  IDs  (see  table  2)  performed  with  the  proposed  decluttered 
HSDH  are  presented  in  figure  20.  The  maximum  mean  value  was  3.63  for  task  ID  14 
performing  a  controlled  termination  to  touchdown  at  LZ4.  No  pilot  reported  a  workload  value 
greater  than  6  for  the  proposed  decluttered  HSDH. 
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Figure  19.  Proposed  HSDH  mean  workload  ratings  for  mission  task  IDs. 
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Figure  20.  Proposed  decluttered  HSDH  mean  workload  ratings  for  mission  task  IDs. 
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3.2.3  Workload  for  Each  Overall  Mission  Task  by  HSDH  Display 

An  analysis  was  also  conducted  between  the  pilots  reported  BWRS  for  overall  mission  ID  and 
the  three  HSDH  designs.  This  analysis  investigated  the  reported  pilot  workload  for  the  overall 
evaluation  tasks.  The  analysis  answers  the  question  of  how  much  workload  did  a  mission  task 
entail  as  compared  across  different  displays.  This  analysis  captures  whether  a  particular  HSDH 
design  has  an  advantage  or  disadvantage  in  a  given  mission  context. 

The  overall  pilot  BWRS  scores  for  mission  task  LZ1  alpha  landing  (see  table  2)  were  2.75,  2.82, 
and  2.55  for  baseline  HSDH,  proposed  HSDH,  and  decluttered  HSDH,  respectively  (see  figure  21). 
Error  bars  represent  the  95%  confidence  interval  of  the  mean.  The  combined  BWRS  scores  for 
LZ1  graphically  appear  statistically  similar,  as  indicated  by  overlap  of  the  95%  confidence 
interval  error  bars.  A  one-way,  repeated  measures  ANOVA  indicated  no  significant  statistical 
difference  among  HSDH  configurations  F  (2,  14)  =  0.16,  p  >  0.05. 


The  overall  pilot  BWRS  scores  for  mission  task  LZ2  hover  to  landing  (see  table  2)  were  4.29, 
3.67,  and  3.29  for  baseline  HSDH,  proposed  HSDH,  and  decluttered  HSDH,  respectively 
(figure  22).  Error  bars  represent  the  95%  confidence  interval  of  the  mean.  The  combined 
BWRS  scores  for  LZ2  graphically  appear  statistically  similar,  as  indicated  by  overlap  of  the  95% 
confidence  interval  error  bars.  A  one-way,  repeated  measures  ANOVA  indicated  no  significant 
statistical  difference  among  HSDH  configurations  F  (2,  14)  =  1 .40,  p  >  0.05. 
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Figure  22.  Workload  ratings  for  overall  mission  task  LZ2  hover  to  land. 


The  overall  pilot  BWRS  scores  for  mission  task  LZ3  sling  load  (see  table  2)  were  3.92,  3.92,  and 
3.30  for  baseline  HSDH,  proposed  HSDH,  and  decluttered  HSDH,  respectively  (figure  23). 

Error  bars  represent  the  95%  confidence  interval  of  the  mean.  The  combined  BWRS  scores  for 
LZ3  graphically  appear  statistically  similar,  as  indicated  by  overlap  of  the  95%  confidence 
interval  error  bars.  A  one-way,  repeated  measures  ANOVA  indicated  no  significant  statistical 
difference  among  HSDH  configurations  F  (2,  14)  =  0.95,  p  >  0.05. 


o 

<J 


m 

0£ 


C 

(D 

<11 


10 

9 

8 

7 

6 

5 

4 

3 

2 

1 


Overall  Workload  for  LZ3 


Baseline 


Proposed 


DeCluttered 


Figure  23.  Workload  ratings  for  overall  mission  task  LZ3  sling  load. 
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The  overall  pilot  BWRS  scores  for  mission  task  LZ4  hover  to  landing  (see  table  2)  were  3.75, 
3.79,  and  3.33  for  baseline  HSDH,  proposed  HSDH,  and  decluttered  HSDH,  respectively 
(figure  24).  Error  bars  represent  the  95%  confidence  interval  of  the  mean.  The  combined 
BWRS  scores  for  LZ4  graphically  appear  statistically  similar,  as  indicated  by  overlap  of  the  95% 
confidence  interval  error  bars.  A  one-way,  repeated  measures  ANOVA  indicated  no  significant 
statistical  difference  among  HSDH  configurations  F  (2,  14)  =  0.44,  p  >  0.05. 
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Figure  24.  Workload  ratings  for  overall  mission  task  LZ4  high  hover  to  land. 

The  overall  pilot  BWRS  scores  for  mission  task  LZ6  reposition  (see  table  2)  were  4.00,  3.375, 
and  3.375  for  baseline  HSDH,  proposed  HSDH,  and  decluttered  HSDH,  respectively  (figure  25) 
Error  bars  represent  the  95%  confidence  interval  of  the  mean.  The  combined  BWRS  scores  for 
LZ4  graphically  appear  statistically  similar,  as  indicated  by  overlap  of  the  95%  confidence 
interval  error  bars.  A  one-way,  repeated  measures  ANOVA  indicated  no  significant  statistical 
difference  among  HSDH  configurations  F  (2,  14)  =  0.54,  p  >  0.05. 
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Figure  25.  Workload  ratings  for  overall  mission  task  LZ6  reposition. 

3.3  Crew  Station  Pilot-Vehicle  Interface 

The  PVI  questionnaire  investigated  pilots’  perceptions  and  preferences  of  the  different  HSDH 
displays  to  inform  them  of  the  successful  control  of  the  aircraft.  Pilots’  comparisons  of  the 
overall  usability  of  the  baseline,  proposed,  and  decluttered  displays  were  collected. 

3.3.1  Overall  PVI  Ratings 

The  overall  crew  station  PVI  questionnaire  responses  for  all  PVI  dimensions  were  1.97,  1.75, 
and  1.73  for  baseline,  proposed,  and  decluttered  HSDHs,  respectively  (figure  26).  Recall  that  a 
PVI  score  between  1  and  2  indicates  that  the  rater  selected  between  “very  easy/very  effective” 
and  “somewhat  easy/somewhat  effective”  on  average.  Error  bars  represent  the  95%  confidence 
interval  of  the  mean.  In  addition,  participant  comments  regarding  general  concerns  about  the 
HSDH  designs  and  the  three  things  they  liked/disliked  about  the  design  of  interface  are  presented 
in  appendix  F. 

3.3.2  PVI  Comparisons  by  HSDH  Display 

A  breakdown  of  the  PVI  responses  by  category,  PV-1,  PV-2,  PV-3,  and  PV-4  for  each  HSDH 
design  is  presented  in  figures  27-29.  Smaller  values  represent  greater  pilot  preference. 

For  the  baseline  HSDH,  the  overall  mean  pilot  report  for  establishing  and  maintaining  hover  is 
1.88,  falling  nearest  to  the  somewhat  effective  category.  The  overall  mean  pilot  rating  for 
supporting  landing  is  1.63.  For  the  baseline  HSDH,  the  mean  pilot  report  for  overall  mission 
effectiveness  is  1.74,  falling  nearest  to  the  somewhat  effective  category.  For  the  baseline  HSDH, 
the  mean  pilot  report  for  maintaining  situational  awareness  is  1.75,  falling  nearest  to  the 
somewhat  effective  category. 
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Figure  26.  Crew  overall  PVI  ratings  by  HSDH  design. 


PVI  ratings  for  baseline  HSDH 

°  ■  Pilot 

4.00 


3.50 


£  3.00 

o 

u 

in 


PV-1  PV-2  PV-3  PV-4 


Figure  27.  PVI  ratings  by  dimension  for  baseline  HSDH. 
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Figure  28.  PVI  ratings  by  dimension  for  proposed  HSDH. 


PVI  ratings  for  decluttered  HSDH 

°  ■  Pilot 

4.00 

3.50 


£  3.00 

o 

u 

t/1 


>  2.50 

c 

ns 

<u 

S  2.00 


1.50 


1.00 


PV-1  PV-2  PV-3  PV-4 


Figure  29.  PVI  ratings  by  dimension  for  proposed  decluttered  HSDH. 
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For  the  proposed  HSDH,  the  overall  mean  pilot  report  for  establishing  and  maintaining  hover  is 
1.13,  falling  nearest  to  the  very  effective  category.  The  overall  mean  pilot  rating  for  supporting 
landing  is  1.38,  falling  nearest  the  very  effective  category.  For  comparison  ratings  between  the 
proposed  HSDH  and  the  baseline  HSDH,  the  mean  pilot  report  for  overall  mission  effectiveness 
is  1.50  in  favor  of  the  proposed  HSDH.  For  comparison  ratings  between  the  proposed  HSDH 
and  the  baseline  HSDH,  the  mean  pilot  report  for  maintaining  situational  awareness  is  1.75  in 
favor  of  the  proposed  HSDH. 

For  the  proposed  decluttered  HSDH,  the  overall  mean  pilot  report  for  establishing  and 
maintaining  hover  is  1.43,  falling  nearest  to  the  very  effective  category.  The  overall  mean  pilot 
rating  for  supporting  landing  is  1.00,  falling  at  the  very  effective  category.  For  comparison 
ratings  between  the  proposed  decluttered  HSDH  and  the  proposed  HSDH,  the  mean  pilot  report 
for  overall  mission  effectiveness  is  1.63  in  favor  of  the  proposed  decluttered  HSDH.  For 
comparison  ratings  between  the  proposed  decluttered  HSDH  and  the  baseline  HSDH,  the  mean 
pilot  report  for  maintaining  situational  awareness  is  1.35  in  favor  of  the  proposed  decluttered 
HSDH.  For  comparison  ratings  between  the  proposed  decluttered  HSDH  and  the  baseline 
HSDH,  the  mean  pilot  report  for  overall  situational  awareness  is  1.49  in  favor  of  the  proposed 
decluttered  HSDH.  For  comparison  ratings  between  the  proposed  decluttered  HSDH  and  the 
proposed  HSDH,  the  mean  pilot  report  for  maintaining  situational  awareness  is  1 .63  in  favor  of 
the  proposed  decluttered  HSDH. 

3.3.3  Pilot  After  Action  Review  Summary 

An  all-hands  discussion  was  conducted  with  the  seven  remaining  pilots  on  the  final  day  of  the 
CSWG  demonstration.  Notes  were  taken  of  the  discussion  by  two  independent  observers.  These 
two  sets  of  notes  were  compared  and  compiled.  A  summary  of  the  main  points  of  the  group 
discussion  is  presented  as  follows. 

•  All  pilots  reported  favorable  preferences  for  the  proposed  and  decluttered  HSDH. 

•  All  pilots  indicated  that  the  proposed  and  decluttered  displays  improved  hover 
performance. 

•  Pilots  indicated  that  the  VSI  tape  as  simulated  was  too  wide  and  obscured  the  VSI  scale. 

•  Pilots  reported  that  the  yellow  VSI  tape  worked  well  for  attention  capture  for  descent  rates 
greater  than  the  landing  limit. 

•  Pilots  reported  that  locating  a  digital  readout  of  the  RAD  ALT  on  the  left  side  of  the  HSD 
would  decrease  the  required  visual  scan  area. 

•  Pilots  discussed  that  the  VSI  tape  helped  them  visually  confirm  a  positive  rate  of  climb. 

•  Pilots  reported  a  mixed  vote  for  and  against  the  hangman  for  hover  altitude  hold  indication. 
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•  Pilots  generally  liked  the  rising  ground  plane  cue,  though  some  reported  that  its  brown 
color  was  lost  in  the  yellow  of  the  VSI  tape. 

•  Pilots  unanimously  favored  the  horizontal  guidance  cue.  “Put-the -ball-in-cup”  became  a 
commonly  repeated  theme  of  the  demonstration.  Some  pilots  did  report  that  scale  changes 
of  the  guidance  cue  was  an  aggressive  approach  compared  to  their  approach  practice. 

•  All  pilots  voted  in  favor  of  immediate  implementation  of  the  horizontal  guidance  cue. 

3.4  Aircraft  Control  Measures 

In  order  to  determine  the  utility  of  the  HSDH  display  designs  to  inform  the  pilots’  maneuvering 
of  the  aircraft,  five  different  aircraft  control  measures  were  defined  and  analyzed  for  specific 
mission  tasks.  These  analyses  allow  for  the  investigation  of  how  the  displayed  cues  influenced 
the  aviating  performance  of  the  pilots.  The  aircraft  control  measures  conducted  are  transverse 
landing  speed,  vertical  speed  at  landing,  horizontal  deviation  in  landing,  horizontal  deviation  in 
vertical  descent,  and  vertical  deviation  during  sling  load  operations.  Due  to  one  pilot  being 
present  for  only  two  of  the  HSD  types,  all  aircraft  control  measures  were  analyzed  on  only  the 
seven  remaining  pilots. 

3.4.1  Transverse  Landing  Speed 

The  speed  of  the  aircraft  perpendicular  to  the  longitudinal  axis  of  the  aircraft  was  measured  for 
all  landings.  Lower  transverse  landing  speeds  result  in  a  lower  risk  of  rollover  on  landing.  The 
graph  of  transverse  landing  speed  by  pilot  can  be  seen  in  figure  30,  and  the  graph  of  transverse 
landing  speed  by  landing  site  can  be  seen  in  figure  31. 

A  one-way  repeated  measures  ANOVA  indicated  no  significant  change  in  transverse  landing 
speed,  F( 2,  6)  =  1.30,  p  >  0.05,  suggesting  that  the  new  symbology  did  not  result  in  lesser  or 
greater  transverse  landing  speeds.  Transverse  landing  speeds  tended  to  be  greater  during  alpha 
approaches  (LZ1,  KBYS). 

3.4.2  Vertical  Speed  at  Landing 

Although  pilots  were  not  instructed  to  land  at  any  specific  or  minimum  speed,  it  is  desirable  that 
the  pilot  land  at  a  speed  below  the  maximum  rate  of  descent  at  touchdown  (landing  limit). 
Landings  above  this  rate  could  result  in  damage  to  the  aircraft.  The  number  of  landings  for 
which  the  vertical  speed  was  above  the  landing  limit  were  tallied  and  can  be  seen  by  pilot  in 
figure  32  and  by  landing  site  in  figure  33.  A  one-way  repeated  measures  ANOVA  indicated  no 
significant  change  in  the  number  of  landings  above  the  maximum  rate  of  descent  at  touchdown, 
F( 2,  6)  =  2.67,  p  >  0.05.  Overall,  there  were  18  rough  landings  with  the  baseline  symbology,  16 
with  the  proposed  symbology,  and  12  with  the  decluttered  symbology. 
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Transverse  Landing  Speed 


Figure  30.  Transverse  landing  speed  by  pilot. 


Transverse  Landing  Speed  by  Landing  Site 


Figure  31.  Transverse  landing  speed  by  landing  site. 
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Figure  32.  Vertical  landing  speed  by  pilot. 
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Figure  33.  Vertical  landing  speed  by  landing  site. 
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The  number  of  rough  landings  was  higher  during  alpha  approaches  (LZ1,  KBYS)  accounting  for 
31  of  a  total  45  rough  landings.  These  31  rough  landings  during  alpha  approaches  were  out  of  42 
possible  alpha  approach  landings  for  an  overall  rough  landing  percentage  of  73.8%  as  compared 
to  14  out  of  63  non-alpha  approach  landings  for  an  overall  percentage  of  22.2%. 

3.4.3  Horizontal  Deviation  During  LZ1  Approach 

The  (x,  y)  position  of  the  aircraft  during  the  approach  to  LZ1  while  in  brownout  conditions  was 
plotted,  and  a  linear  regression  was  performed.  The  coefficient  of  determination  for  this  linear 
regression  was  calculated  to  give  a  measure  of  the  linearity  of  the  approach  in  the  (x,  y)  plane. 
Larger  coefficients  of  determination  would  indicate  a  more  linear  approach,  indicating  a  more 
controlled  approach.  The  coefficients  of  determination  by  pilot  for  LZ1  can  be  seen  in  figure  34. 
Two  of  the  pilots  had  large  oscillations  when  close  to  the  ground  during  one  of  the  runs  which 
were  not  representative  of  an  alpha  approach.  The  graph  of  the  remaining  pilots  data  can  be  seen 
in  figure  35. 

A  one-way  repeated  measures  ANOVA  indicated  no  significant  change  in  coefficient  of 
determination,  F( 2,  6)  =  0.72,  p  >  0.05,  suggesting  that  the  new  symbology  did  not  result  in  a 
straighter  path  during  alpha  approach.  The  ANOVA  for  the  reduced  group  of  pilot  data  did  not 
show  a  significant  change  either,  F(2,4)  =  2.18,  p  >  0.05. 


Coefficient  of  Determination  by  Pilot  for  LZ1 
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Figure  34.  Coefficient  of  determination  by  pilot  for  LZ1. 
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Coefficient  of  Determination  by  Pilot  for  LZ1 
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Figure  35.  Coefficient  of  determination  by  pilot  for  LZ1  with  outliers  removed. 


3.4.4  Horizontal  Deviation  During  Vertical  Descent 

The  (x,  y)  position  of  the  aircraft  during  descent  at  LZ4  while  in  brownout  conditions  was 
recorded,  and  the  mean  position  was  calculated.  From  this,  the  average  linear  deviation  in  feet 
during  the  descent  was  calculated  to  provide  a  measure  of  how  close  the  approach  was  to 
perfectly  vertical.  Lower  values  indicate  that  the  approach  was  more  vertical.  A  graph  of  the 
results  by  pilot  can  be  seen  in  figure  36. 

A  one-way  repeated  measures  ANOVA  indicated  no  significant  change  in  the  coefficient  of 
determination,  F( 2,  6)  =  2.36,  p  >  0.05,  suggesting  that  the  new  symbology  did  not  result  in  a 
more  vertical  descent.  Overall,  deviation  during  vertical  descent  was  not  large  in  any  condition. 
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Average  Horizontal  Deviation  from  Mean 
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Figure  36.  Horizontal  deviation  from  mean  during  vertical  descent  at  LZ4. 


Additionally,  the  deviation  in  feet  from  the  planned  landing  zone  location  and  the  actual  location 
at  which  the  aircraft  was  landed  was  calculated.  Results  can  be  seen  in  figure  37.  A  one-way 
repeated  measures  ANOVA  indicated  a  significant  change  in  coefficient  of  determination,  F( 2, 

6)  =  4.30,  p  =  0.039,  indicating  that  the  new  symbology  allowed  the  pilot  to  more  accurately  land 
the  aircraft  at  the  desired  landing  site. 
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Deviation  from  Desired  LZ  at  landing 
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Figure  37.  Horizontal  deviation  from  desired  LZ4  at  landing. 


3.4.5  Vertical  Deviation  During  Sling  Load  Operations 

The  average  deviation  in  feet  around  the  desired  15 -ft  hover  height  for  the  sling  load  operation 
was  calculated.  The  beginning  and  end  of  the  15-ft  hover  were  determined  by  the  analyst’s 
judgment.  Where  possible,  the  inflection  point  in  the  altitude  data,  which  indicated  a  controlled 
stop  to  the  descent,  was  used  as  the  start  point,  and  the  last  point  before  ascent  was  used  as  the 
end  point.  Lower  values  indicate  that  the  aircraft  altitude  was  maintained  closer  to  the  desired 
15-ft  level  during  this  section  of  the  operation.  The  average  error  by  pilot  can  be  seen  in 
figure  38. 

A  one-way  repeated  measures  ANOVA  indicated  a  significant  change  in  coefficient  of 
determination,  F( 2,  6)  =  5.71, p  =  0.018,  indicating  that  the  new  symbology  allowed  the  pilot  to 
more  accurately  control  the  vertical  position  of  the  aircraft  during  a  hover. 
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Average  Altitude  Error  for  15'  Sling  Load 
Operation 
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Figure  38.  Average  altitude  error  for  15-ft  sling  load  operation. 


3.5  Training  Assessment 

Pilots  were  surveyed  on  their  perceptions  of  the  effectiveness  of  the  training  session  conducted 
on  day  one  of  the  CSWG  demonstration.  The  mean  overall  training  assessment  rating  for  all 
participants  was  3.6,  i.e.,  the  mean  response  was  between  the  “agree”  and  “strongly  agree” 
responses.  Higher  scores  are  better.  The  training  results  are  further  detailed  by  each  question  in 
figure  39. 
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Figure  39.  Overall  training  assessment  ratings  by  question. 
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4.  Discussion 


4.1  Visual  Gaze 

Acknowledging  the  small  sample  size,  the  data  pattern  for  fixation  durations,  which  can  be 
viewed  as  short  processing  cycles,  indicate  that,  for  the  evaluation  maneuvers  considered  as  a 
whole,  the  display  type  had  no  consistent  effect  on  pilots’  fixations.  As  for  the  implications  to 
workload,  the  two  proposed  displays  did  not  increase  workload  as  compared  to  the  existing 
display.  This  is  an  important  metric  for  “doing  no  harm”  when  designing  new  user  interfaces. 

No  reduction  in  workload  for  the  fixation  duration  measure  was  found  for  the  new  display 
designs.  This  may  be  attributed  to  the  lack  of  training  with  this  particular  simulator,  or  the  small 
sample  size. 

The  interfixation  angle  was  reduced  in  the  proposed  and  decluttered  conditions  compared  to  the 
baseline  condition.  This  is  most  likely  due  to  the  smaller  visual  search  areas  needed  to  gather  the 
required  information  to  control  the  aircraft.  These  data  indicate  a  pattern  showing  that  pilots 
moved  their  eyes  shorter  distances  for  the  new  display  designs.  This  effect,  though  not 
statistically  significant  in  likelihood  due  to  the  small  sample  size,  is  most  likely  due  to  the 
reduced  scan  area  of  the  new  displays  as  compared  to  the  baseline.  Although  the  following 
figure  is  not  a  representation  of  the  entire  sample  see  figure  40,  this  illustrates  the  expected  and 
exhibits  in  at  least  one  instance,  a  scanning  pattern  that  would  be  performed  in  a  real  aircraft  in 
which  pilots  have  more  experience  with  the  proposed  HSDH  symbology. 

As  can  be  seen  in  the  error  bars  for  fixation  frequency  (figure  12),  pilots  exhibited  a  high  degree 
of  individual  differences  for  fixation  frequency  patterns.  This  participant  variability  is  an 
obvious  mitigating  factor  for  identifying  a  decrease  in  fixation  frequency  and  a  concomitant 
increase  in  information  gained  per  fixation. 

The  blink  frequency  was  reduced  in  the  proposed  and  decluttered  conditions  compared  to  the 
baseline  condition.  A  tenuous  inverse  relationship  has  been  shown  to  exist  between  blink 
frequency  and  workload  measures.  Blink  frequency  may  indeed  be  associated  with  higher  visual 
workload  which,  in  the  case  of  the  proposed  and  decluttered  HSDH,  may  be  due  to  the  higher 
information  availability  in  a  smaller  scan  area.  Additionally,  blinks  are  often  conducted  before 
saccades.  With  a  significant  reduction  in  interfixation  angle,  it  is  likely  that  the  opportunity  to 
perform  efficient  blinks  was  not  presented  as  frequently  with  the  proposed  and  decluttered 
HSDH  displays. 
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Figure  40.  Comparison  of  common  visual  scanning  patterns. 
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The  percentage  of  visual  gaze  in  the  HSD  AOI  was  increased  in  the  proposed  and  decluttered 
conditions  compared  to  the  baseline  condition,  while  the  percentage  of  visual  gaze  in  the  VSI 
AOI  was  decreased.  This  is  most  likely  caused  by  the  relocation  of  vertical  speed  information  to 
the  vertical  speed  tape,  which  is  located  in  the  HSD  AOI.  The  percentage  of  visual  gaze  in  the 
RAD  ALT  AOI  stayed  relatively  constant,  which  correlates  well  with  pilots  comments  during 
After  Action  Review  that  they  were  often  looking  for  the  RAD  ALT  Digital  Readout  to  get  the 
exact  height  in  feet.  Pilots’  visual  gaze  patterns  were  advantageously  restructured  with  the  two 
new  HSDH  displays.  Ocular  motion  was  concentrated  in  a  smaller  area  of  the  HSD  AOI.  This 
means  that  a  pilot’s  total  visual  attention  was  released  from  necessary  gaze  patterns  over  to  the 
VSI. 

The  collective  results  from  the  HSD  and  RAD  ALT  gaze  analyses  favorably  support  the 
proposed  HSDH  design.  During  the  collective  after  action  review,  pilots  openly  discussed  the 
synergy  between  the  proposed  HSD  and  RAD  ATL.  Pilots  recommended  locating  the  digital 
RAD  ALT  value  closer  to  the  HSD  horizontal  acceleration  cue.  The  concentrated  gaze  on  the 
new  HSD  displays  and  continued  reliance  on  RAD  ALT  indicate  that  new  useful  information  for 
the  controlling  the  horizontal  disposition  of  the  aircraft  was  available  to  pilots  in  the  proposed 
and  decluttered  HSDH  displays.  Please  see  appendix  F  for  overall  percentages  of  time  spent 
gazing  at  the  RAD  ALT,  HSD,  and  VSI  across  each  display  condition. 

The  observed  gaze  patterns  for  the  VSI  AOI  mean  that  a  pilot  significantly  saves  both  time  and 
ocular  distance  travelled  on  the  proposed  HSDH  designs  as  compared  to  the  baseline  HSDH. 

For  statistical  significance,  note  that  the  error  bars  in  figure  16,  defined  as  confidence  intervals, 
for  the  proposed  and  decluttered  HSDH  do  not  overlap  with  the  baseline  HSDH  error  bars. 

4.2  Workload 

From  the  perspective  of  pilot  mental  workload,  the  various  HSDH  displays  performed  similarly 
across  different  mission  tasks.  Overall  average  subjective  workload  scores  ranged  in  the  3  to  4 
range,  well  below  the  critical  value  of  6.  The  workload  scores  indicated  that  the  evaluation 
maneuvers  were  challenging,  but  did  not  detrimentally  overwhelm  pilots’  available  spare 
workload  capacity.  Each  of  the  HSDH  displays  supported  the  successful  control  of  the  aircraft 
and  successful  execution  of  the  mission  tasks.  Furthermore,  no  single  display  showed  an 
advantage  toward  reducing  workload  for  any  of  the  particular  mission  tasks  or  subtasks. 

4.3  Crew  Station  Pilot-Vehicle  Interface 

The  PVI  questionnaire  indicated  a  high  degree  of  pilot  acceptance  for  the  proposed  and 
decluttered  HSDH  displays.  The  overall  crew  station  PVI  questionnaire  responses  for  all  PVI 
dimensions  indicated  that  the  pilots  selected  between  “very  easy/very  effective”  and  “somewhat 
easy/somewhat  effective”  on  average.  For  the  baseline  HSDH,  the  overall  mean  pilot  report  for 
establishing  and  maintaining  hover  fell  nearest  to  the  somewhat  effective  category. 
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For  the  proposed  HSDH,  the  overall  mean  pilot  report  for  establishing  and  maintaining  hover  fell 
nearest  to  the  very  effective  category.  The  overall  mean  pilot  rating  for  supporting  landing  fell 
nearest  the  very  effective  category.  For  comparison  ratings  between  the  proposed  HSDH  and  the 
baseline  HSDH,  the  mean  pilot  report  for  overall  mission  effectiveness  was  in  favor  of  the 
proposed  HSDH.  For  comparison  ratings  between  the  proposed  HSDH  and  the  baseline  HSDH, 
the  mean  pilot  report  for  maintaining  situational  awareness  was  in  favor  of  the  proposed  HSDH. 

For  the  proposed  decluttered  HSDH,  the  overall  mean  pilot  report  for  establishing  and 
maintaining  hover  fell  nearest  to  the  very  effective  category.  The  overall  mean  pilot  rating  for 
supporting  landing  fell  at  the  very  effective  category.  For  comparison  ratings  between  the 
proposed  decluttered  HSDH  and  the  proposed  HSDH,  the  mean  pilot  report  for  overall  mission 
effectiveness  was  in  favor  of  the  proposed  decluttered  HSDH.  For  comparison  ratings  between 
the  proposed  decluttered  HSDH  and  the  baseline  HSDH,  the  mean  pilot  report  for  maintaining 
situational  awareness  was  in  favor  of  the  proposed  decluttered  HSDH.  For  comparison  ratings 
between  the  proposed  decluttered  HSDH  and  the  baseline  HSDH,  the  mean  pilot  report  for 
overall  situational  awareness  was  in  favor  of  the  proposed  decluttered  HSDH.  For  comparison 
ratings  between  the  proposed  decluttered  HSDH  and  the  proposed  HSDH,  the  mean  pilot  report 
for  maintaining  situational  awareness  was  in  favor  of  the  proposed  decluttered  HSDH. 

4.4  Aircraft  Control  Measures 

The  absolute  error  of  the  aircraft  when  landing  at  a  particular  LZ  was  greatly  reduced  due  to  the 
changes  incorporated  in  the  Proposed  HSDH.  The  horizontal  speed  guidance  encourages  the 
pilot  to  bring  the  aircraft  directly  to  the  landing  zone  or  hover  reference  point.  The  pilots  in  this 
study  referred  to  the  changing  of  the  magenta  horseshoe  to  a  white  circle  as  “the  circle  of 
confidence,”  which  indicated  to  them  that  they  were  at  a  controlled  hover  at  the  desired  location. 
This  resulted  in  pilots  more  accurately  locating  the  aircraft  over  the  desired  LZ. 

The  altitude  of  the  aircraft  was  more  controlled  during  hover  operations  due  to  changes 
incorporated  in  the  proposed  HSDH.  The  hangman  gives  the  pilot  an  easy  to  follow  reference  in 
order  to  adjust  his  torque  to  establish  or  maintain  a  specific  altitude.  The  movement  of  this 
hangman  provides  immediate  feedback  to  the  pilot  on  any  variation  in  altitude. 

The  transverse  landing  speed  as  a  function  of  display  type  was  not  statistically  significant.  This 
may  be  partly  due  to  the  fact  that  the  transverse  landing  speeds  were  not  particularly  great  due  to 
the  nature  of  the  various  landings.  Landings  that  were  mostly  vertical  did  not  have  much 
variance  in  any  direction  and  were  generally  controllable.  It  may  be  more  valuable  to  study  the 
amount  of  landings  that  were  over  a  certain  safe  transverse  velocity  value  in  a  larger  set  of 
landings  to  determine  if  the  new  display  types  were  helpful  in  landings  that  for  one  reason  or 
another  were  not  as  smooth.  The  transverse  landing  speeds  tended  to  be  higher  on  the  angled 
approaches  than  on  the  vertical  approaches.  This  may  be  due  to  the  fact  that  all  horizontal 
velocities  are  generally  higher  on  angled  approaches. 
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The  number  of  landings  above  the  landing  limit  was  not  statistically  significant.  This  may  be 
due  in  part  to  the  fact  that  the  pilots  were  not  trying  to  control  for  minimum  or  very  slow  landing 
speeds.  Furthermore,  the  difficulty  of  controlling  the  torque  of  aircraft  in  the  simulator  may  have 
resulted  in  overcompensation. 

The  coefficient  of  determination  for  the  landing  at  LZ1  was  not  statistically  significant.  This  was 
due  to  the  fact  that  most  of  the  landings  at  LZ1  had  very  little  deviation  from  a  straight  line  path. 
The  consistency  of  performance  indicates  that  there  was  not  much  room  for  improvement  with 
the  proposed  displays.  The  few  landings  that  deviated  from  a  straight  line  path  were  large 
deviations  unassociated  with  the  display  technology.  For  instance,  the  largest  deviation  came 
from  one  pilot  when  he  tried  to  come  to  a  hover  near  to  the  ground  at  the  end  of  the  angled 
approach  even  though  this  maneuver  was  not  part  of  the  mission  requirements. 

The  average  deviation  from  the  mean  horizontal  position  during  the  vertical  approach  to  LZ4 
was  not  statistically  significant.  The  average  deviation  was  relatively  small  in  all  attempts, 
indicating  that  this  type  of  approach  is  well  controlled  in  the  baseline  HSDH  condition,  leaving 
little  room  for  improvement. 

4.5  Training  Assessment 

Participants  rated  the  overall  training  effectiveness  with  a  mean  of  3.6  on  a  1—4  scale,  where  4 
represented  “strongly  agree”  regarding  the  effectiveness  of  the  training.  The  training  was  well 
received  by  participants.  However,  it  could  have  been  longer  with  more  pre-test  simulator  time 
given  to  reduce  learning  and  order  effects  during  the  demonstration. 


5.  Recommendations 


Within  the  CSWG  demonstration,  four  lines  of  evidence  converge  indicating  a  high  degree  of 
pilot  acceptance  for  the  overall  design  and  implementation  of  the  proposed  and  decluttered 
HSDH  displays.  Crew  gaze  analyses  indicate  pilots  had  reduced  scanning  extents  for  the  VSI 
AOI  in  the  proposed  and  decluttered  HSDH  as  compared  to  the  baseline  HSDH.  Thus,  the  new 
HSDH  should  reduce  visual  workload.  The  smaller  footprint  of  the  information  required  for 
hover  and  landing  as  presented  in  the  new  HSDH  should  aid  the  pilots  when  they  perform  these 
tasks.  Mental  workload  measures  verified  the  challenging  nature  of  the  simulated  brownout 
DVE  landing  while  also  recommending  the  new  HSDH  displays  for  consuming  slightly  less 
spare  workload  capacity  of  the  pilots  than  the  baseline  display.  Aircraft  control  analyses  also 
recommend  the  new  HSDH  designs  for  effectively  supporting  the  aviating  performances  of  the 
pilots.  Open-ended  pilot  comments  suggested  making  the  horizontal  speed  cue  less  aggressive  in 
order  to  alleviate  the  abrupt  effect  associated  with  the  auto-scaling.  The  comments  of  pilots  also 
recommend  that  the  RAD  ALT  digital  readout  be  located  near  to  the  vertical  speed  tape,  to 
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give  actual  height  when  the  altitude  information  is  needed.  Additionally,  pilots  recommended 
making  vertical  speed  tape  change  color  only  on  conditions  such  as  over  the  landing  limit. 
Unanimously,  pilots  voted  in  favor  of  the  immediate  implementation  of  the  proposed  HSDH 
display. 
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Appendix  A.  Demographic  Questionnaire 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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CH-47  Hover  Symbology  CSWG  Demographic  Data  Sheet 

The  information  requested  on  this  form  will  be  used  only  for  the  purpose  of  obtaining  a 
profile  of  those  pilots  who  complete  the  training  and  demonstration  missions.  Your 
responses  will  remain  anonymous.  Names  will  be  dissociated  from  the  reported  data.  If 
you  are  uncomfortable  answering  any  question,  please  omit  it.  Any  information 
provided  will  be  used  solely  as  part  of  the  Army  Research  Laboratory  efforts  to  gather 
feedback  on  the  CH-47  Hover  Symbology.  Thank  you  in  advance  for  completing  the 
demographic  data  sheet. 

Name: _ (Please  Print) 

Confidentiality  PIN: _  Current  Unit: _ 

Rank: _  Date  of  Rank: _ 

Your  MOS: _ 

What  is  your  primary  aircraft? _ 

Dominant  Hand:  Right  /  Left  (circle  one) 

Sight  correction  while  flying:  Yes  /  No  Correction  type: _ 
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Gender: 


Male  or  Female 


Age  Group  (years;  circle  one): 

Under  20 
20-24 
25-29 
30-34 
35-40 

Education  Level  (circle  one): 

High  School/GED 

1- 2  yrs  college 
Associates  Degree 
Bachelor's  Degree 
Masters  Degree 

Service  Component  (circle  one): 

Active  Army 
Reserve 
National  Guard 

Other  Service: _ 

Military  Service  Duration  (circle  one): 

2- 5  years  1 5-20  years 

6-9  years  20-25  years 

1 0-1 5  years  Over  25  years 


40-45 

45  and  over 
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Airframe  Experience: 


Platform 

Qualified 

IP  Hours 

PI  Hours 

PIC  Hours 

NVD  Hours 

CH-47 

Other  (name) 

Other  (name) 

48 


Appendix  B.  Bedford  Workload  Rating  Scale  (BWRS)  Questionnaire 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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For  Official  Use  Only 
CH-47  I1SDII  Workload 

Name:  Date: 

1.  Crew  # _ 

2.  W  hich  HSDH  was  used?  Baseline  Proposed  Proposed  Decluttered  (Circle  One) 

Workload 

3.  Refer  to  the  Bedford  Workload  Rating  Scale  included  in  this  packet.  The  Bedford  scale  employs  the 
concept  of  snare  capacity  to  measure  mental  workload.  Based  on  that  scale  and  its  descriptions,  rate  the 
workload  for  the  Bight  and  mission  tasks  you  performed  during  the  previous  mission  scenario. 

For  example,  if  when  Operating  the  CH-47  you  feel  that  “reduced  workload  capacity”  is  an  accurate 
description  for  your  spare  rapacity  on  that  task,  please  indicate  a  ‘5’  in  the  Bedford  Workload  Rating 
Column. 


Flight  and  Mission  Tasks 

Bedford  Workload 

1  =  Lowest  to  10  =  Highest 

Land  at  smoke  in  field  (overall) 

Maintain  constant  approach  to  desired  landing  point 

Maintain  rate  of  closure 

Maintain  ground  track  alignment 

Perform  a  controlled  termination  to  touchdown 

Land  at  mosque  (overall) 

Perform  a  controlled  termination  to  initial  hover 

Perform  a  controlled  descent  with  minimal  drift 

Perform  sling  load  operations  (overall) 

Perform  a  controlled  termination  to  initial  hover 

Perform  a  controlled  termination  to  hookup  point 

Land  at  confined  area  (overall) 

Perform  a  controlled  termination  to  initial  hover 

Perform  a  controlled  descent  with  minimal  drift 

Relocate  landing  point  (overall) 
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For  Official  Use  Only 

CH-47  HSDH  Workload 

If  you  gave  a  workload  rating  of  ‘6’  or  higher  for  any  task,  explain  why  the  workload  was  high  for  that 
task: 


lit  the  mission  you  just  flew,  list  any  flight  and/or  mission  tasks  that  you  had  to  ask  your  crewmember  to 
accomplish  because  your  workload  was  too  high:  (Use  hack  for  additional  space). 
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Workload  insignificant 

1 

Workload  low 

2 

Enough  workload  capacity  for  all 
desirable  additional  tasks 

3 

Is  workload  tolerable  I  THT  I 

fertile  task?  I  NU  I 


Insufficient  workload  capacity  for  easy 
attention  to  additional  tasks 


Reduced  workload  capacity.  Additional 
tasks  cannot  be  given  the  desired  amount 
of  attention 


Little  workload  capacity  level  of  effort 
allows  little  attention  to  additional  tasks 


Very  little  spare  capacity,  but 
maintenance  of  effort  in  the  primary 
tasks  not  in  question 


Very  high  workload  with  almost  no 
spare  capacity  Difficulty  in  maintaining 
level  of  effort 
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Appendix  C.  Pilot  Vehicle  Interface  (PVI)  Questionnaires 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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PILOT-VEHICLE  INTERFACE  (P\  I)  QUESTIONNAIRE 
Baseline  HSDH 

1.  PIN:  2.  Date: 

3.  Crew  #  _ 

The  information  provided  will  be  used  for  the  purpose  of  providing  feedback  related  to  the  CH- 
47  Baseline  HSDH.  Your  responses  should  be  based  only  on  symbology  used  during  the  mission 
that  you  just  completed. 

PV1.  Rate  the  effectiveness  of  these  Hover  Symbology  items  for  supporting  the  following 
mission  objectives. 

Hover  Box  for  Establishing  Desired  Hover  Altitude 


1 

2 

3 

4 

(Circle  One) 

Very 

Effective 

Comments: 

Somewhat 

Effective 

Somewhat 

In-Effective 

Very 

In-Effective 

Hover  Box  for  Maintaining  Desired  Altitude 

1 

2 

3 

4 

(Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

Radar  Altitude  Indicator  for  Landing 

1 

2 

3 

4 

(Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 
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Radar  Altitude  Indicator  for  Establishing  Desired  Altitude 

12  3  4 


Very  Somewhat  Somewhat  Very 

Effective  Effective  In-Effective  In-Effective 

Comments: _ 

Radar  Altitude  Indicator  for  Maintaining  Desired  Altitude 

12  3  4 


Very  Somewhat  Somewhat  Very 

Effective  Effective  In-Effective  In-Effective 

Comments: 


Hover  Vertical  Speed  Scale  for  Controlling  Vertical  Speed 

12  3  4 

(Circle  One) 

Very 

Effective 

Comments: 

Somewhat 

Effective 

Somewhat 

In-Effective 

Very 

In-Effective 

Overall  Baseline  IISDII  for  Establishing/Maintaining  Hover 

1 

2 

3 

4 

(Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

Overall  Baseline  HSDH  for  Handing 

1 

2 

3 

4 

(Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 
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PV2.  Please  rate  the  following  properties  of  the  Baseline  HSDH 

Size/Location  of  the  Hover  Box 


1 

2 

3 

4  (Circle  One) 

Excellent 

Comments: 

Good 

Poor 

Unsatisfactory 

Size/Location  of  the  Torque  Tape 

1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Size/Location  of  the  Airspeed  Tape 

1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Size/Loeation  of  the  Barometric  Altitude  Tape 

1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Copyright  ©  Jared  Sapp  &  Dr.  Anthony  Morris  2010 


56 


PV3.  Please  rate  the  ease  of  understanding  the  color  schemes  on  tire  Baseline  HSDH: 


1 

2 

3 

4 

(Circle  (One) 

Very 

Somewhat 

Somewhat 

Very 

Easy 

Easy 

Difficult 

Difficult 

Was  your  mission  task  affected  by  the  HSDH  color  scheme?  Yes  No  (Circle  One) 

If  yes,  then  please  explain: 


PV4.  Please  rate  the  Baseline  HSDH  with  respect  to  overall  mission  effectiveness. 


1 

2 

3 

4  (Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

Please  rate  the  Baseline  HSDH  with  respect  to  maintaining  situational  awareness. 


1 

2 

3 

4  (Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

Copyright  ©  Jared  Sapp  &  Dr.  Anthony  Morris  2010 


57 


1. 
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PILOT-VEHICLE  INTERFACE  (P\  I)  QUESTIONNAIRE 
Proposed  HSDH 

1.  PIN:  2.  Date: 

3.  Crew  #  _ 

The  information  provided  will  be  used  for  the  purpose  of  providing  feedback  related  to  the  CH- 
47  Proposed  HSDH.  Your  responses  should  be  based  only  on  symbology  used  during  the 
mission  that  you  just  completed. 

PV1.  Rate  the  effectiveness  of  these  Hover  Symbology  items  for  supporting  the  following 
mission  objectives. 

Vertical  Speed  Indicator  Tape  for  Establishing  Desired  Hover  Altitude 


1 

2 

3 

4 

(Circle  One) 

Very 

Effective 

Comments: 

Somewhat 

Effective 

Somewhat 

In-Effective 

Very 

In-Effective 

Vertical  Speed  Indicator  Tape  for  Maintaining 

Desired  Altitude 

1 

2 

3 

4 

(Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

V  ertical  Speed  Indicator  Tape  for  Landing 

1 

2 

3 

4 

(Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

Copyright  ©  Jared  Sapp  &  Dr.  Anthony  Morris  2010 


59 


Vertical  Speed  Indicator  Tape  for  Controlling  Vertical  Speed 


1 

2 

3 

4  (Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

Target  Horizontal  Speed  Guidance  Horseshoe  for  Control  Guidance  to  Desired 


Hover/Landing  Point 

1 

2 

3 

4 

(Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

High  Hover  Indicator  (Hangman)  for  Controlling  Hover  Altitude 

12  3  4 

Very  Somewhat  Somewhat  Very 

Effective  Effective  In-Effective  In-Effective 

Comments: _ 

Overall  Proposed  HSDH  for  Establishing/Maintaining  Hover 

12  3  4 

Very  Somewhat  Somewhat  Very 

Effective  Effective  In-Effective  In-Effective 

Comments : _ 

Overall  Proposed  HSDH  for  Landing 


1 

2 

3 

4  (Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 
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PV2.  Please  rate  the  following  properties  of  the  Proposed  HSDH 


Size/Location  of  the  Vertical  Speed  Tape 

1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Size/Location  of  the  Ground/High  Hover  Indicator  (Hangman)  on  Vertical  Speed  Tape 


1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Size/Location  of  the  Target  Horizontal  Speed  Guidance  Horseshoe 


1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Scale  of  the  Vertical  Speed  Tape  for  Vertical  Speed 


1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Scale  of  the  Vertical  Speed  Tape  for  Altitude 


1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 
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PV3.  Please  rate  the  ease  of  understanding  the  color  schemes  on  the  Proposed  HSDH: 

12  3  4 

_  (Circle  One) 

Very  Somewhat  Somewhat  Very 

Easy  Easy  Difficult  Difficult 

Was  your  mission  task  affected  by  the  HSDH  color  scheme?  Yes  No  (Circle  One) 

If  yes,  then  please  explain: 


PV4. 

Please  rate  the  Proposed  HSDH  compared  to  the  Baseline  HSDH  with  respect  to  overall 


mission  effectiveness. 

1  2 

3 

4 

(Circle  One) 

Much  More 

Somewhat  More 

Somewhat  Less 

Much  Less 

Effective 

Effective 

Effective 

Effective 

Comments: 

Please  rate  the  Proposed  HSDH  compared  to  the  Baseline  HSDH  with  respect  to  maintaining 


situational  awareness. 

1  2 

3 

4 

(Circle  One) 

Much  More 

Somewhat  More 

Somewhat  Less 

Much  Less 

Effective 

Effective 

Effective 

Effective 

Comments: 
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PV5. 

Please  list  two  (2)  things  you  liked  about  the  Proposed  HSDH: 


1. 

2. 

Please  list  two  (2)  things  you  did  not  like  about  the  Proposed  HSDH: 

1. 

2. 

Please  provide  other  comments  you  have  not  covered  elsewhere: 
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PILOT  VEHICLE  INTERFACE  (PVI)  QUESTIONNAIRE 
Proposed  Decluttered  HSDH 

1.  PIN:  2.  Date: 

3.  Crew  #  _ 

The  information  provided  will  be  used  for  the  purpose  of  providing  feedback  related  to  the  CH- 
47  Proposed  Decluttered  HSDH.  Your  responses  should  be  based  only  on  symbology  used 
during  the  mission  that  you  just  completed. 

PVI.  Rate  the  effectiveness  of  these  Hover  Symbology  items  for  supporting  the  following 
mission  objectives. 

Vertical  Speed  Indicator  Tape  for  Establishing  Desired  Hover  Altitude 


1 

2 

3 

4 

(Circle  One) 

Very 

Effective 

Comments: 

Somewhat 

Effective 

Somewhat 

In-Effective 

Very 

In-Effective 

Vertical  Speed  Indicator  Tape  for  Maintaining  Desired  Altitude 

12  3  4 

(Circle  One) 

Very 

Effective 

Comments: 

Somewhat 

Effective 

Somewhat 

In-Effective 

Very 

In-Effective 

Vertical  Speed  Indicator  Tape  for  Landing 


1 

2 

3 

4  (Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

Ei-Effective 

Comments: 
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Vertical  Speed  Indicator  Tape  for  Controlling  Vertical  Speed 


1 

2 

3 

4  (Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

Target  Horizontal  Speed  Guidance  Horseshoe  for  Control  Guidance  to  Desired 


Hover/Landing  Point 

1  2 

3 

4 

(Circle  One) 

Very 

Effective 

Comments: 

Somewhat 

Effective 

Somewhat 

In-Effective 

Very 

In-Effective 

High  Hover  Indicator  (Hangman)  for  Controlling  Hover  Altitude 

1 

2 

3 

4 

(Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

Overall  Proposed  Decluttered  HSDH  for  Establishing/Maintaining  Hover 

1 

2 

3 

4 

(Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 

Overall  Proposed  Decluttered  HSDH  for  Landing 

1 

2 

3 

4 

(Circle  One) 

Very 

Somewhat 

Somewhat 

Very 

Effective 

Effective 

In-Effective 

In-Effective 

Comments: 
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PV2.  Please  rate  the  following  properties  of  the  Proposed  Decluttered  HSDH 


Size/Location  of  the  Vertical  Speed  Tape 

1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Size/Location  of  the  Ground/High  Hover  Indicator  (Hangman)  on  Vertical  Speed  Tape 


1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Size/Location  of  the  Target  Horizontal  Speed  Guidance  Horseshoe 


1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Size/Location  of  the  Torque  Readout 


1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Size/Location  of  the  Airspeed  Readout 


1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 
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Scale  of  the  Vertical  Speed  Tape  for  Vertical  Speed 


1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

Scale  of  the  Vertical  Speed  Tape  for  Altitude 

1 

2 

3 

4  (Circle  One) 

Excellent 

Good 

Poor 

Unsatisfactory 

Comments: 

PV3.  Please  rate  the  ease  of  understanding  the  color  schemes  on  the  Proposed  Decluttered 
HSDH: 

1 

2 

3 

4 

(Circle  One) 

Very 

Easy 

Somewhat 

Easy 

Somewhat 

Difficult 

Very 

Difficult 

Was  your  mission  task  affected  by  the  HSDH  color  scheme?  Yes  No  (Circle  One) 

If  yes,  then  please  explain: 
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PV4. 

Please  rate  the  Proposed  Decluttered  HSDH  compared  to  the  Baseline  HSDH  with  respect  to 


overall  mission  effectiv  eness. 

1  2 

3 

4 

(Circle  One) 

Much  More 

Somewhat  More 

Somewhat  Less 

Much  Less 

Effective 

Effective 

Effective 

Effective 

Comments: 

Please  rate  the  Proposed  Decluttered  HSDH  compared  to  the  Proposed  HSDH  with  respect  to 


overall  mission  effectiveness. 

1  2 

3 

4 

(Circle  One) 

Much  More 

Somewhat  More 

Somewhat  Less 

Much  Less 

Effective 

Comments: 

Effective 

Effective 

Effective 

Please  rate  the  Proposed  Decluttered  HSDH  compared  to  the  Baseline  HSDH  with  respect  to 


maintaining  situational  awareness. 

12  3 

4 

(Circle  One) 

Much  More 

Somewhat  More 

Somewhat  Less 

Much  Less 

Effective 

Effective 

Effective 

Effective 

Comments: 

Please  rate  the  Proposed  Decluttered  HSDH  compared  to  the  Proposed  HSDH  with  respect  to 


maintaining  situational  awareness. 

12  3 

4 

(Circle  One) 

Much  More 

Somewhat  More 

Somewhat  Less 

Much  Less 

Effective 

Effective 

Effective 

Effective 

Comments: 
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PV5. 

Please  list  two  (2)  things  you  liked  about  the  Proposed  Decluttered  HSDH: 


1. 

2. 

Please  list  two  (2)  things  you  did  not  like  about  the  Proposed  Decluttered  IISDII: 

1. 

2. 

Please  provide  other  comments  you  have  not  covered  elsewhere: 
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Intentionally  lelt  blank. 
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Appendix  D.  Summary  of  Pilot  Vehicle  Interface  (PVI)  Comments 


This  appendix  appears  in  its  original  form,  without  editorial  change. 


Baseline  HSDH 


PV-1  Comments 

•  Hover  Box  for  Establishing  Desired  Hover  Altitude 

o  Quite  honestly  my  brain  doesn’t  even  identify  that  it’s  there, 
o  Will  be  cross  checked  with  RAD  ALT. 

o  I  don’t  use  it  because  it  gives  no  real  idea  of  where  the  ground  is. 

o  Did  not  look  at  it  for  altitude  reference,  the  hover  pose  was  too  cluttered  when  +  or 
-  10  ft  of  the  hover  reference  altitude  the  box  looked  the  same. 

o  Not  analyzed 

•  Hover  Box  for  Maintaining  Desired  Altitude 
o  Will  be  crossed  checked  with  RAD  ALT 
o  Not  analyzed 

•  Radar  Altitude  Indicator  for  Landing 
o  One  of  my  primary  cues 

o  Graphic  depiction  would  be  better  then  reading  numbers 

•  Radar  Altitude  Indicator  for  Establishing  Desired  Altitude 

•  Radar  Altitude  Indicator  for  Establishing  Desired  Altitude 
o  Had  to  use  the  USI  to  maintain  altitude 

•  Hover  Vertical  Speed  Scale  for  Controlling  Vertical  Speed 
o  Combined  solution  with  RAD  ALT  indicator. 

o  I  still  tend  to  look  at  the  VSD  on  the  upper  half  of  the  MFD,  just  because  that’s  what  is 
referenced  most  of  the  time  in  forward  flight. 

o  It’s  effective  however:  it  creates  to  large  of  a  scan  area. 

o  Lound  myself  looking  at  the  vertical  speed  next  to  the  VSD. 
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•  Overall  Baseline  HSDH  for  Establishing/Maintaining  Hover 
o  Used  it  for  years  with  great  success. 

o  Works  better  with  PHOLD 

o  The  symbology  is  spread  out  too  far  on  the  MFD,  a  lot  information  that  wasn’t  needed, 
o  I  used  the  VSD  for  pitch  indication. 

•  Overall  Baseline  HSDH  for  Landing 

o  Once  an  established  hover  is  attainted,  landing  is  less  difficult  than  maintaining  the 
hover. 

o  VSD  used  for  pitch. 

PV-2  Comments 

•  Size  and  Location  of  the  Hover  Box 
o  Hard  to  rate  since  I  do  not  utilize, 
o  I  feel  it  could  be  smaller. 

o  I  don’t  like  it,  not  as  sensitive  too.  Give  me  accurate  information, 
o  Not  Analyzed 

•  Size/Location  of  the  Torque  Tape 

o  However  I  do  not  use  tape;  I  only  focus  on  digits, 
o  Location  creates  too  large  of  a  scan  area, 
o  Didn’t  look  at  it  for  the  hovering, 
o  I  don’t  look  at  the  tape.  Just  the  number. 

•  Size/Location  of  the  Airspeed  Tape 
o  Primarily  use  ground  speed. 

o  G  speed  could  be  moved  up  to  rod  indicator  line, 
o  Ground  speed  is  what  needed  when  landing  in  brown-out. 

o  Didn’t  look  at  it,  used  the  GS  lead-out  during  the  transition  to  hover  and  for  hovering. 

•  Size/Location  of  the  Barometric  Altitude  Tape 
o  Not  used  during  brown-out. 
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o  Didn’t  look  at  it. 


o  I  primarily  use  RAD  ALT. 

PV-3  Comments 

•  Was  mission  affected  by  HSDH  color  scheme?  (If  yes,  please  explain) 
o  Red  is  difficult  to  read  at  night. 

PV-4  Comments 

•  Overall  Mission  Effectiveness 

o  Scanning  the  large  area  can  cause  drift. 

•  Maintaining  Situational  Awareness 

o  Would  like  to  have  a  cue  for  next  way  point  regardless  of  Flight  Plan  sequence. 

o  The  hover  reference  was  useful  because  you  can  see  your  intended  hover  point  and 
landing  direction. 

PV-5  Comments 

•  Liked 

o  Ease  of  interpretation 
o  Regardless  of  its  change  or  not  it  is  usable, 
o  Familiarity  of  use 
o  Drift  Vectors 

o  Symbol  change  when  changing  from  PH/TRC 
o  Hover  reference  home  plate 
o  Hover  modes,  Auto,  Trans,  etc. 
o  Shows  neutral  stick  position 
o  Where  hover  reference  is  to  you  location 
o  RAD  ALT 
o  Velocity  vector 

o  Auto- scanning  was  very  useful/reduce  workload 
o  Good  organization  and  density  of  provided  data 
o  10  kt/30  kt  ring 
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Dislike 


o  HURBox 
o  Dog  house 

o  Too  much  info  when  not  decluttered 
o  Hover  dot  and  the  reference  about  the  same  size 
o  Hover  box 

o  Airspeed/Distance  ring  only  changes  number  value  when  going  to  the  next  scale,  needs 
to  have  color  change  of  size  of  ring  change. 

o  Placement  of  VSI,  and  GS 

o  Too  much  info  not  needed  for  hovering/dust  landing 
o  Needs  more  info  due  to  fixation  on  instruments 
o  Altitude  Readout 
o  Torque  Readout 

o  Altitude/RAD  ALT  was  somewhat  out  of  “field  of  view” 
o  Found  hover  cues  to  be  initially  counter-intuitive 
o  Torque  Tape 
o  Vertical  situating  horizon 

•  Comments  not  covered  elsewhere 

o  Adapting  to  HSDH  was  not  difficult  but  required  more  than  one  practice  session. 
(Subject  has  no  47F  experience  or  Training) 

Proposed  HSDH 
PV-1  Comments 

•  Vertical  Speed  Indicator  Tape  for  Establishing  desired  Hover  Altitude 

o  If  the  tape  could  be  a  bit  more  narrow.  It  tends  to  cover  up  the  altitude  tick  marks. 

o  The  only  time  I  really  devoted  any  attention  to  it  was  when  it  covered  the  G/S.  Other 
than  that,  I  was  more  apt  to  use  the  VSI  that  I  am  used  to. 

o  At  first,  a  little  hard  to  decipher  rate  of  descent. 
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o  When  I  looked  at  it! 


Vertical  Speed  Indicator  Tape  for  Maintaining  Desired  Altitude 
o  Went  back  to  old  scan  once  established  to  the  altimeters  on  the  right  of  MFD 
o  Easily  noticed  change  from  climb  and  descent, 
o  The  color’s  changing  cued  me  to  check  my  rate  of  descent. 

Vertical  Speed  Indicator  Tape  for  Landing 

o  Very  nice  gives  a  wake-up  call.  However,  may  cause  Pilots  to  start  reaching  for  the 
ground.  Power  in,  power  out,  in,  out 

o  When  descent  was  enough  to  trigger  yellow  bar,  it  was  difficult  to  determine  rate  of 
descent;  yellow  bar  over-powered  scale. 

Vertical  Speed  Indicator  Tape  for  Controlling  Vertical  Speed 

o  I  liked  the  color  change  when  the  rate  become  excessive. 

Target  Horizontal  Speed  Guidance  To  Desired  Hover/Landing  Point 

o  Loved  It!  It  forces  Pilots  to  maintain  that  slow  forward  movement  to  target.  This 
feature  will  also  be  very  useful  for  pinnacles  from  an  OGE  hover  with  no  ground 
references. 

o  Very  easy  to  decipher 

o  Was  effective  if  your  hover  page  was  active  prior  to  0.2  Nm,  gave  me  time  to  react 

o  Sometimes,  I  would  remember  that  I  would  also  have  to  slide  left  or  right  depending  on 
my  landing  heading. 

High  Hover  Indicator  (Hangman)  for  Controlling  Hover  Altitude 

o  I  don’t  know  if  I  ever  looked  at  it  for  landing.  Too  much  info  on  one  small  area. 
Altitude  lines  hangman  vertical  speed  indicator  etc. 

o  Hangman  bleached  out  by  yellow  bar 

o  It  blended  into  the  background;  the  V.S.  tape  overpowered  the  hangman  and  washed  it 
out. 

o  My  eyes  seem  to  just  scan  over  it,  but  I  was  more  interested  in  Rad  Altitude 


•  Overall  Proposed  HSDH  for  Establishing/Maintaining  Hover 

o  Less  hangman-  can  we  make  the  brown  line  go  all  the  way  across  the  HSDH.  The  long 
Brown  line  would  stay  level,  giving  you  good  reference  off  the  HSDH  roll  indicator, 
not  having  to  reference  USD  up  top. 

•  Overall  Proposed  HSDH  for  Landing 

o  I  like  knowing  where  the  ground  is. 

PV-2  Comments 

•  Size/Location  of  the  Vertical  Speed  Tape 
o  Size  is  too  wide.  Would  like  it  narrow, 
o  Wish  scale  was  bigger. 

•  Size/Location  of  the  Ground/High  Hover  Indicator  (Hangman)  on  Vertical  Speed  Tape 

o  To  thick,  didn’t  like  it  at  all.  Right  handed  look  right.  I  want  a  hard  number.  Also 
brown  may  be  hard  to  interpret  under  goggles  with  a  dim  cockpit.  Too  much  clutter. 

o  It  blended  into  the  background,  needs  to  be  a  little  larger  or  maybe  not  associated  with 
the  V.S.  tape. 

•  Size/Location  of  the  Target  Horizontal  Speed  Guidance  Horseshoe 
o  Keep  it. 

•  Scale  of  the  Vertical  Speed  Tape  for  Vertical  Speed 
o  Again  more  narrow. 

o  Shouldn’t  cover  any  other  data  at  any  time, 
o  Scale  hard  to  read  with  bar  and  hangman  over  top  of  scale. 

•  Scale  of  the  Vertical  Speed  Tape  for  Altitude 
o  Needs  to  be  bigger. 

o  Not  used;  the  V.S.  tape  dominated  that  portion  of  my  scan. 

PV-3  Comments 

•  Was  your  mission  task  affected  by  the  HSDH  color  scheme?  If  yes,  then  please  explain, 
o  Do  not  like  anything  brown  in  the  cockpit  color  scheme. 

o  Color  of  hangman  does  not  stand  out  enough;  yellow  bar  very  distracting  and  too 
bright. 
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o  The  yellow  color  of  the  V.S.  tape  was  too  bright;  captured  a  good  portion  of  my  scan. 

o  I  would  catch  my  rate  of  descent  through  the  comer  of  my  eye  and  was  able  to  react  to 
the  color  yellow. 

o  It  informed  me  of  excessive  rates  of  descent. 

PV-4  Comments 

•  Proposed  HSDH  compared  to  Baseline  HSDH  (Overall  Mission  Effectiveness) 

o  I  don’t  know  if  relearning  new  cues  over  and  over  again  is  actually  counterproductive 
to  building  and  improving  on  cues  already  in  place;  small  additions  (i.e.,  the 
horseshoes)  are  good  but  the  need  to  completely  change  the  VSI  is  just  not  needed. 

o  Smaller  scan. 

•  Proposed  HSDH  compared  to  Baseline  HSDH  (Maintaining  Situational  Awareness) 
o  For  two  reasons,  the  Horseshoe  and  the  vertical  speed  tape. 

o  Nice  to  know  where  the  ground  is. 

PV-5  Comments 

•  Likes 

o  Horseshoe 
o  Vertical  Speed  Tape 
o  Horseshoe 
o  Hover  reference 
o  Guidance  cup 
o  Heading  bug 

o  The  horseshoe  was  very  effective, 
o  The  vertical  speed  tape 

o  VSI  color — easy  to  know  if  your  rate  of  descent  was  too  high 
o  Smaller  scan 
o  Vertical  speed  rate 
o  Hangman 

o  Improved  situational  awareness  (ref:  altitude  and  vertical  speed) 
o  Better  direction  to  attain/maintain  hover  point 
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o  VSItape 
o  THSG  (horseshoe) 

•  Dislikes 

o  Hangman 

o  Any  brown  colors  of  MFD 

o  The  “hangman”  combined  with  the  VSI/RALT  is  too  much  info  in  one  spot  to  be  really 
useful,  that  and  with  the  fact  that  the  RALT  (proposed)  has  no  hard  number  with  it 
made  it  unusable  to  me.  The  interpretation  of  a  scaled  VSI  in  a  DVE  situation  causes  a 
lag  in  response  where  seeing  a  hard  Gist  number  on  the  legacy  VSI/RALT  takes  one 
less  computation  for  my  brain  to  make  resulting  in  a  faster  response. 

o  VSI  tape  scale 

o  Hangman  color 

o  Hangman 

o  The  .2  Nm  sequence  to  hover  point  was  too  close  to  hover  point 
o  Hangman 

o  Ball  cup  when  changing  levels.  It  seemed  that  I  needed  to  put  in  more  cyclic  since  the 
cup  changed  scales.  But  the  cup  started  to  move  much  faster. 

o  Cup  movement  when  AUTO  is  selected  (didn’t  like  it  snapping  into  different  scales). 

o  Found  that  increased  VSI/ground  awareness  may  lead  to  overcontrolling  thrust. 

o  Found  horizontal  speed  guidance  cues  a  bit  aggressive. 

o  CHEAC  thrust  issues  complicated  the  assessment  (control  model  inaccurate). 

o  Horizontal  line  on  horizontal  situation 

o  Torque  tape 

Decluttered  HSDH 
PV-1  Comments 

•  Vertical  Speed  Indicator  Tape  for  Establishing  desired  Hover  Altitude 

o  The  color  caught  my  attraction  but  I  still  reverted  to  legacy  VSI  for  majority  of  time, 
o  Good  Location  too  wide  when  it’s  solid 
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o  Scale  hard  to  read 


•  Vertical  Speed  Indicator  Tape  for  Maintaining  Desired  Altitude 
o  Couldn’t  keep  myself  from  using  legacy  VSI. 

o  I  still  find  myself  using  altimeter  on  right. 

•  Vertical  Speed  Indicator  Tape  for  Landing 

o  Couldn’t  keep  myself  from  using  legacy  VSI. 
o  Still  using  Altimeter 
o  Yellow  bar  too  bright 

o  Gave  good  visual  reference  of  the  rate  of  descent, 
o  The  more  and  more  I  use  this,  I  am  able  to  pick  up  on  the  information. 

•  Vertical  Speed  Indicator  Tape  for  Controlling  Vertical  Speed 
o  Great  for  a  wake-call  for  descent  when  it  turns  yellow. 

o  Color  scheme  was  good  indicator  of  vertical  speed. 

•  Target  Horizontal  Speed  Guidance  To  Desired  Hover/Landing  Point 
o  Love  it!  Many  different  ways  to  use  it. 

•  High  Hover  Indicator  (Hangman)  for  Controlling  Hover  Altitude 

o  Never  used  the  legacy  cues  are  so  easy  for  me  to  use  that  I  basically  disregarded  the 
hangman. 

o  I  just  don’t  use  it.  I’m  sure  it  works.  I  just  don’t  use  it. 
o  Color  needs  to  stand  out  more. 

o  Used  today  seemed  like  it  was  more  pronounced  versus  the  proposed. 

o  The  scale  seems  like  it  would  get  you  close  to  the  height,  but  a  solid  number  is  better 
for  a  hover. 

•  Overall  Proposed  HSDH  for  Establishing/Maintaining  Hover 
o  Very  clean  picture.  Not  confusing. 

o  Loved  the  GS  location. 

o  Could  move  up  the  radar  alt.  readout  higher  to  line  up  the  lateral  scan, 
o  Very  simple. 
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•  Overall  Proposed  HSDH  for  Landing 
o  Less  the  hangman. 

o  Decluttered  had  enough  info  to  do  the  task. 

PV-2  Comments 

•  Size/Location  of  the  Vertical  Speed  Tape 

o  Size  and  location  are  really  not  an  issue, 
o  Size  of  scale  needs  to  be  bigger. 

•  Size/Location  of  the  Ground/High  Hover  Indicator  (Hangman)  on  Vertical  Speed  Tape 
o  Rising  ground  indicator  may  be  more  useful  if  stretched  through  the  whole  HSI. 

o  Could  be  a  little  more  pronounced  larger  scale  or  brighter  color. 

•  Size/Location  of  the  Target  Horizontal  Speed  Guidance  Horseshoe 

•  Size  Location  of  the  Torque  Readout 
o  Torque  tape  decluttered 

•  Size  Location  of  the  Ground  Speed  Readout 

o  Would  like  to  see  maybe  one  or  two  fonts  larger  for  clarity, 
o  Really  didn’t  use. 

•  Scale  of  the  Vertical  Speed  Tape  for  Vertical  Speed 
o  Still  like  real  numbers. 

o  Too  small. 

•  Scale  of  the  Vertical  Speed  Tape  for  Altitude 

o  Will  take  some  getting  use  to,  but  it  can  be  affective.  I  still  prefer  a  hard  number  for  the 
altimeter. 

o  Too  small. 

PV-3  Comments 

•  Ease  of  understanding  the  color  schemes.  If  yes,  then  please  explain. 

o  Color  of  hangman  blends  in — yellow  bar  bleaches  out  VSI  scale. 

o  The  yellow  arrow  for  rate  of  descent,  I  feel  like  as  soon  as  it  turned  into  the  arrow  I 
needed  a  lot  more  thrust  than  I  actually  needed. 
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PV-4  Comments 


•  Proposed  Decluttered  HSDH  compared  to  Baseline  HSDH  (Overall  Mission  Effectiveness) 
o  Simpler  the  better. 

o  Less  Hangman. 

o  Ease  of  use  not  as  busy  looking. 

o  It  had  enough  information  I  needed  for  the  task. 

•  Proposed  Decluttered  HSDH  compared  to  Proposed  HSDH  (Overall  Mission 
Effectiveness) 

o  Because  of  location  of  GS . 

o  All  the  info  on  the  left  side  of  HSDH,  all  easier  to  read  and  understand. 

•  Proposed  Decluttered  HSDH  compared  to  Baseline  HSDH  (Maintaining  Situational 
Awareness) 

•  Proposed  Decluttered  HSDH  compared  to  Proposed  HSDH  (Maintaining  Situational 
Awareness) 

o  A  lot  less  confusing. 

PV-5  Comments 

•  Likes 

o  Simplicity 
o  Less  confusion 
o  GS  location 
o  Torque  location 
o  Ground  Speed  Readout 

o  Unnecessary  information  was  removed  to  ease  the  scan, 
o  I  like  the  TQ  readout  without  the  tape. 

o  Liked  having  GS  readout  on  the  same  side  as  the  TQ  and  HS  Tape 
o  Plenty  of  information 

o  Less  confusing  with  too  much  going  on  with  the  page, 
o  Less  Busy 
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o  Less  Redundancy 
•  Dislikes 
o  Hangman 
o  Proposed  VSI 

o  Radar  Altitude  Readout  was  too  low,  for  ease  of  lateral  scan  can  be  lined  up  on  the 
same  plane  as  GS  readout. 

o  It  would  be  nice  to  have  the  RAD  ALT  on  the  same  side  as  the  TQ  and  GS. 
o  I  don’t  like  having  RAD  ALT  scale  of  VSI  on  the  HSDH 
o  Radar  Alt  scale.  The  number  is  all  I  look  at. 
o  Horizon  line  on  horizontal  situation. 
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Intentionally  lelt  blank. 
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Appendix  E.  Pilot  Training  Assessment  Questionnaire 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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The  training  I  received... 


Strongly 

Disagree 

1 


Disagree 

2 


Agree 

3 


Strongly 

Agree 

4 


1.  The  practice  flights  were 
helpful. _ 


2.  The  duration  of  the 
training  and  practice 
flights  was  adequate. 

1 

2 

3 

4 

3.  The  training  covered  the 
important  aspects  of  the 
new  Hover  Symbology. 

1 

2 

3 

4 

4.  Overall,  the  training  1 
received  was  adequate  for 
the  CSWG  event. 

1 

2 

3 

4 

5.  1  am  confident  1  can  fly  the 
CSWG  Missions  with  the 
information  and  training 
provided. 

1 

2 

3 

4 

Copyright  ©  Jared  Sapp  &  Dr.  Anthony  Morris  2010 
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Figure  F-l.  Pilot  visual  gaze  distribution  for  baseline  Horizontal  Situation  Display  Hover  (HSDH). 
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Figure  F-2.  Pilot  visual  gaze  distribution  for  proposed  HSDH. 
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Pilot  Outboard  -  CH47F 
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Figure  F-3.  Pilot  visual  gaze  distribution  for  decluttered  HSDH. 
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Appendix  G.  Summary  of  Training  Survey  Comments 


Positive  Comments: 


None  Reported 


Negative  Comments: 

•  None  Reported 
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Intentionally  lelt  blank. 
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Appendix  H.  Summary  of  Bedford  Workload  Rating  Scale  Comments 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Q:  If  you  gave  a  workload  rating  of  '6'  or  higher  for  any  task,  explain  why  the  workload  was 
high  for  that  task: 

A:  Smoke  field-  That  landing  area  best  suits  a  running  landing.  No  references  at  all  make  it 
more  difficult.  Mosque-  obstacles  require  controlled  drift.  Sling  loads-  Trying  to  be  precise  not 
bang  up  equipment.  Confined  area-  obstacles 

A:  Land  at  Mosque:  It  was  difficult  to  determine  vertical  speed  and  maintain  alt./position  of 
hover  part.  As  I  looked  at  VSI I  would  lose  my  scan  on  the  altitude  and  the  hover  reference. 

A:  Just  know  my  surroundings.  I  noticed  my  right  pedal  and  tried  to  convert  it.  But  fixation  on 
trim  -ball  oversaturated  the  maneuvers.  Hover  symbology  was  not  in  my  mind. 

A:  Field-  No  references.  Pilot  knows  he  will  hit  dust  making  the  anticipation  higher  for  the 
landing.  Confined  Area-  Tough  with  obstacles  everywhere,  no  PHOLD 

A:  New  scan:  Reading  and  understanding  the  cues  was  difficult. 

Q:  In  the  mission  you  just  flew,  list  any  flight  and/or  mission  tasks  that  you  had  to  ask  your 
crewmember  to  accomplish  because  your  workload  was  too  high:  (Use  back  for  additional 
space). 

A:  Co-Pilot  called  out  Airspeed  and  Altitude  as  part  of  normal  crew  coordination. 

A:  Normal  Crew  interaction 

A:  Normal  Crew  Coordination  per  ATM  (Aircrew  Training  Manual) 

A:  Normal  Crew  Coordination  call-outs 

A:  Back  me  up  on  RAD  ALT  during  hover  operations.  I  found  myself  focusing  on  getting  the 
ball  in  the  cup. 

A:  Basic  Crew  Coordination  IAW  CH-47  Aircrew  Training  Manual 
A:  Normal  Crew  call-outs 

A:  Normal  Crew  Coordination  IAW  Aircrew  Training  Manual 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


AAR  After  Action  Review 

AED  Aviation  Engineering  Directorate 

AMRDEC  Aviation  and  Missile  Research,  Development,  and  Engineering  Command 

ANOVA  Analysis  of  Variance 

AOI  Area  of  Interest 

ASL  Applied  Science  Laboratories 

ATM  Aircrew  Training  Manual 

BHIVE  Battlefield  Highly  Immersive  Virtual  Environment 
BWRS  Bedford  Workload  Rating  Scale 

CAAS  Common  Aviation  Architecture  System 

CDU  Control  Display  Unit 

CH-EAC  Cargo  Helicopter  -  Engineering  Analysis  Cockpit 

Cl  Confidence  Interval 

CSWG  Crew  Station  Working  Group 

DVE  Degraded  Visual  Environments 

fpm  Feet/Minute 

GS  Ground  Speed 

HFE  Human  Factors  Engineering 

HSD  Horizontal  Situation  Display 

HSDH  Horizontal  Situation  Display  Hover 

HSI  Horizontal  Speed  Indicator 

IAW  In  Accordance  With 

IP  Instructor  Pilot 

Bicycle  Lake  Army  Airfield 


KBYS 
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LZ1 

Landing  Zone  1 

MFD 

Multi-Function  Display 

NTC 

National  Training  Center 

PH 

Position  Hold 

PHOLD 

Position  Hold 

PVI 

Pilot  Vehicle  Interface 

RAD  ALT 

Radar  Altitude 

RALT 

Radar  Altimeter 

SA 

Situational  Awareness 

SME 

Subject  Matter  Expert 

SSDD 

System  Simulation  and  Development  Directorate 

TQ 

Torque 

TRC 

Translation  Rate  Command 

TRC 

Translation  Rate  Command 

VFR 

Visual  Flight  Rules 

VS 

Vertical  Speed 

VSD 

Vertical  Situation  Display 

VSI 

Vertical  Speed  Indicator 
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